Oh - we're actually in the process of moving all docs over to this repo: https://github.com/cucumber/docs.cucumber.io
I would say environment variables or the world parameters CLI option: https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#world-parameters
Just yesterday I discovered the multiple interfaces that may be used with Cucumber-JS step definitions (https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/step_definitions.md). The asynchronous promise interface seems cleaner than the asynchronous callback interface. Can someone with more experience describe the pros and cons of each? I'd like to make a relatively informed decision before going too far down either path.
no problem found it! https://github.com/cucumber/cucumber-js/issues/937
Latest docs on the matter: https://github.com/cucumber/cucumber-js/blob/v3.0.3/docs/support_files/attachments.md
There is a snippets formatter for this purpose: https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#formats
cucumber-expressions 4.0.4 is out with some bugfixes: https://github.com/cucumber/cucumber/blob/master/cucumber-expressions/CHANGELOG.md#404---2017-10-05
I started a proof of concept with protractor, angular, and cucumberjs. See if https://github.com/gdinwiddie/AngularCucumber helps at all.
It's quite similar to that: https://github.com/wswebcreation/protractor-cucumber-typescript-boilerplate
Hi. I'm new to Cucumber(.js) + BDD. Looking at line 32 + 36, there's no callback, and it seems common with other examples. http://cucumber.github.io/cucumber-js/. This is my version and I'm using callbacks https://pastebin.com/Lsk9MVyj. is either method wrong or better or .. ?
*Thread Reply:* @txm "This page is no longer available. It has either expired, been removed by its creator, or removed by one of the Pastebin staff."
*Thread Reply:* @gdinwiddie thanks. I'm not 100% sure what it was, I inherited a few samples, I introduced a nice mix of callbacks and Promises. I've set SELENIUMPROMISEMANAGER=0, removed the callbacks and am returning Promises everywhere (without catch) and am trusting them to do their thing and touch wood making good progress.
*Thread Reply:* try adding browser.driver.sleep for 1000ms as first step in every stepdefinition and see if you can actually view some execution happening on the screen
https://github.com/wswebcreation/protractor-cucumber-typescript-boilerplate https://github.com/igniteram/protractor-cucumber-typescript
*Thread Reply:* i am not using proctractor but i had the same issue. What helped was return and chain the promises together.
Hi everyone, I’m working on getting some front end coverage around a legacy site before we start working on more back end functionality. The issue I’m having is that most actions perform page reloads to process interactions. I’m using the example from https://github.com/igniteram/protractor-cucumber-typescript, but I’m trying to figure out the best way to wait for the page to finish loading (like document.ready) befor checking for my ‘Then’ conditions.
There is also: http://webdriver.io/
*Thread Reply:* Sorry, I got the wrong end of the stick.
The new scenario object has a different structure: I had to change this line to read:
After(function (scenarioResult) {
if (scenarioResult.result.status !== 'passed') {
...
Or equally, you could write:
if (scenarioResult.result.status == 'failed') {
...
I worked this out by using console.log(JSON.stringify(scenarioResult)); - YMMV
See updated examples here for checking if a scenario failed in a hook: https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/attachments.md
I'm also seeing this one. It seems that the World object isn't available to the AfterAll hook - which is annoying if you e.g., want to log out at the end of everything...
*Thread Reply:* Problem solved: AfterAll runs after all scenarios.
https://github.com/cucumber/cucumber-js/issues/896
*Thread Reply:* It was the version of the cucumber... the version that worked fine to me was: "cucumber": "~1.3.2", "protractor": "^5.1.1", "protractor-cucumber-framework": "^3.0.0"
*Thread Reply:* @barbara.cabral did you search the GitHub issues for protractor-cucumber-framework to see if anyone else has the same problem?
Hello experts! 🙂 Does anyone know how to basically achieve AfterStep hooks for recent cucumber-js , 3.2.0 ? based on this link, https://github.com/cucumber/cucumber-js/issues/891, there is something called setDefinitionFunctionWrapper that should be able to accomplish the wanted behaviour, but I do not understand it. Does anyone has any example that can achieve the same? The use case is rather simple. I want to verify that the web application is not throwing any browser console error during each step.
*Thread Reply:* I'm trying to solve problems with this new version. But without sucess...
*Thread Reply:* https://www.sitepoint.com/bdd-javascript-cucumber-gherkin/ This walkthrough includes the examples syntax
*Thread Reply:* You’ll probably want to swap to something react-specific or at least not-protractor. Protractor can be used with non-angular frameworks, but I found it a bit clunky compared to just using something framework-agnostic
*Thread Reply:* I read somewhere that provided you switched off the synchronization, it was ok to use Protractor, and also if we put in the Expected Conditions in the test to "work out" when to start.
*Thread Reply:* I don't know of any react specific for this. What did you use in the end?
*Thread Reply:* Webdriverio - works across frameworks + mobile and electron apps
*Thread Reply:* Automation: the game of “How ~lazy~ efficient can I possibly be?”
think i have found the version i need https://github.com/cucumber/cucumber-js/pull/893
Hi, I hope it is okay to post this help request here on Slack, my apologies otherwise.
I have been trying to test IE 11 using Protractor and the Cucumber Test Framework (via Cucumber and protractor-cucumber-framework) on a Windows 10 laptop. So far this hasn't worked despite numerous attempts. Yet when I switch the test framework over to Jasmine it all starts working. We want to use Cucumber for our system test automation program. Full details; including copies of the example project code files; can be seen at: https://stackoverflow.com/questions/49261946/protractor-v5-3-0-protractor-cucumber-framework-v4-2-0-not-working-with-ie11-o
Either way many thanks for looking...
cucumber-js has issues with IE 11. See https://github.com/cucumber/cucumber-js/issues/1011. Does it work in other browsers?
I found my mistake, thanks!
See here for docs on BeforeAll / AfterAll and why this is not a world instance: https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/hooks.md#beforeall--afterall. If you want to share something like a browser between all scenarios, you need to handle that yourself. Cucumber does not have specific support for that because it wants to encourage users to keep scenarios independent so they cannot effect each other. If all scenarios in a feature have some shared precondition use a Background block in your gherkin.
I think this might be the answer. Gonna give it a go. https://stackoverflow.com/a/44373830/226118
import / export are not currently supported by node and thus for cucumber to support them, it would need to transpile the support code. We leave it to the user to define what transpilers to use. https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#transpilers
@charlierudolph committed that line April 14, 2016 - https://github.com/cucumber/cucumber-js/commit/cd68105df2fa3853362b6000c9dad7511d3b7aa4
*Thread Reply:* so, are you saying put assertions and tests in a helper file?
*Thread Reply:* yup. a good example of this is the cucumber-wdio boilerplate (I know you’re using jest, but seeing how they’ve constructed the helpers may be useful for you) https://github.com/webdriverio/cucumber-boilerplate Note that this looks like its builds are failing (I suspect since wdio-cucumber-framework was only recently updated to 2.1), so just use it as POC for grokking the concept for now 🙂
*Thread Reply:* and look at how they’re implemented in src/steps -> src/features
*Thread Reply:* After a quick look at the readme, I again bump into point of confusion. It’s steps have very implementation-specific definitions like Given the checkbox "([^"]**)?" is( not)** checked.... I’ve been informed that this is an anti-pattern and should be avoided in these files. What is your experience with this?
*Thread Reply:* I think it really depends on the specific thing you’re testing. If you have a UI toggle and you need to test what happens if toggled on vs. toggled off, it’s not a bad way to go - because then it’s not so much implementation specific as expressing specific conditions that need to be met prior to performing an action + expected outcome
*Thread Reply:* I also think it depends on how many different elements you have that would have completely different selectors but very similar descriptions (like you have 75 different checkboxes that all have totally different selectors and no common classes or anything)
*Thread Reply:* thanks again for the input. as long as you're willing to help, i'll be pinging you for it. 😆 At least until the fog clears. I stare at my step-def files and go "now what?" 🤷
*Thread Reply:* Yes - just starting out trying to get Jest to work with cucumber - using Jest-cucumber. Working good so far
*Thread Reply:* https://www.npmjs.com/package/jest-cucumber
*Thread Reply:* that's what i'm using as well. Learning multiple tools at once is quite harrowing. Like swimming through quicksand.
*Thread Reply:* Oh, this is gonna be rough. I'm getting syntax errors that have totally unhelpful message. 🤦♂️
@dmshamonov Read about world here https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/world.md
*Thread Reply:* I think that error comes from trying to parse a .feature file as a .js file. Can you check your configuration for what files are being loaded as if they were javascript?
*Thread Reply:* My testMatch mirrors the recommended config "testMatch": ["****/**.steps.js", "****/**.feature"]
*Thread Reply:* I don’t know the jest config. I would look for support on the jest side (wherever the cucumber plugin lives)
My message had no takers 😞
*Thread Reply:* the internal hooks changed a lot in 3.0. What is your use case for needing the feature names?
*Thread Reply:* Thanks for the response Charlie, I run Ui tests through the framework and use browserstack as my remote server.
The framework previously would allow me to get feature names at the start so that when I run tests in parallel using node processes , I could easily mark each feature by the Browser it is run on and post to Browserstack.
That way I name my browserstack session runs which are per feature in my case. Now I know you’d say its not ideal, but it was a conscious decision to run each feature file on a single selenium session and update app state per scenario so that the scenarios were still independent of each other.
*Thread Reply:* That’s not possible anymore. The closest thing you can get is the parameter passed to a before hook. See the api reference doc for more info
I read this but not found answer for my question. I decided what World saves its state only in one Feature context.
*Thread Reply:* Not sure. I’m throwing out ideas here. But if you want to set the environment for each feature, you’ll have to generate some kind of state.
*Thread Reply:* I’ll defer to others that have more experience with this. Also, try the regular #help channel. That may get the answer you need.
*Thread Reply:* @dmshamonov Have you tried using hooks w/ tags to solve your problem? https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/hooks.md
*Thread Reply:* You can do a 'Before' to set the state you desire before any scenarios are ran from within a tagged feature. Otherwise I believe the world object is replaced with each scenario (because no scenario should rely on a previous scenario)
*Thread Reply:* @Ed Snodgrass Ok... In hook "Before" or "Background" I init some data for "Scenarios". Example I create some entity there and save IDs in "World". And I need to delete those entity in end. But in "After" hook "World" isn't available...
*Thread Reply:* @dmshamonov, When I set a value on world, i can print out its contents in the after hook.
//steps file
const { Given, After } = require('cucumber');
After({ tags: '@LogWorldAfterScenario' }, function() {
console.log(this.value);
});
Given(/^I have set a value on the world$/, function() {
this.value = 'Hello World';
});
#feature file
@LogWorldAfterScenario
Feature: After Hook should have the world object
Scenario: Test After Hook
Given I have set a value on the world
*Thread Reply:* if the values are set on the world aren't they being cleaned up after each scenario runs?
fyi, figured out my issue — console.log()!! https://github.com/cucumber/cucumber-js/blob/58caad38be582490ebb1ad4c8e9fe01b2732ace4/docs/cli.md#parallel-experimental
Note: Printing to stdout (using console.log or other means) will cause an error, because the slave processes communicate with the master process over stdout. Instead print to stderr (using console.error or other means).
*Thread Reply:* thanks for responding, @charlierudolph. here’s a snippet of that stack trace:
/Users/scottthompson/WebstormProjects/mx-vtx-ui-enterprise/nodemodules/cucumber/lib/formatter/helpers/eventdata_collector.js:103 if (!step.attachments) { ^
TypeError: Cannot read property ‘attachments’ of undefined at EventDataCollector.storeTestStepAttachment (/Users/scottthompson/WebstormProjects/mx-vtx-ui-enterprise/nodemodules/cucumber/lib/formatter/helpers/eventdata_collector.js:103:16) at emitOne (events.js:115:13) at EventEmitter.emit (events.js:210:7)
*Thread Reply:* Looking at the code, I’m guessing that could happen if you attach something (visa https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/attachments.md) after the step/hook is complete. Can you share the code that attaches something?
*Thread Reply:* After(function (scenario) {
const world = this;
if (scenario.result.status === Status.FAILED) {
if (config.env !== 'dev') scenario.result.exception.stack = null;
// eslint-disable-next-line no-undef
driver.takeScreenshot().then(screenshot => {
world.attach(screenshot, 'image/png');
if (config.env !== 'dev') driver.quit();
});
}
driver.quit();
});
*Thread Reply:* Hmm okay. Can you refactor it so the hook so it returns a promise? Its important that cucumber knows the hook is still running. Currently cucumber would move on before the hook finishes running. I can make the cucumber code a bit more defensive so it gives a nicer error message when this happens in the future
*Thread Reply:* okaaay… i get what your saying about timing out and hookage, but…
*Thread Reply:* The cucumber code keeps in internal stepIndex to know what step to attach to when world.attach is called. Since this call happens after the hook finishes, cucumber has incremented the stepIndex to a non-existent step
*Thread Reply:* i see then too why you mentioned updating error messaging also — to be more specific about WHEN that dies
*Thread Reply:* https://github.com/cucumber/cucumber-js/pull/1081
*Thread Reply:* I’m confused by your question. Is there a javascript equivalent to the cucumberJS method: they would be one and the same.
*Thread Reply:* yea, i'm pretty fried right now. sorry. I meant a pure javascript equivalent. rowsHash() is part of the cucumber-js API. jest-cucumber apparently doesn't support it. I want to take the Feature data table and transform it to a JSON object. Does that help explain?
*Thread Reply:* I do not know jest cucumber. I’d try to ask that question on the jest-cucumber support channels
Just having a bit of trouble understanding how the Definition Function Wrapper is intended to be invoked: https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/step_definitions.md#definition-function-wrapper
It's reminding me of a post I made over eight years ago: https://blog.binarymist.net/2009/12/24/keeping-encapsulation-on-ones-mind/
Don't we have AfterStep hooks now: https://github.com/cucumber/cucumber/wiki/Hooks#tagged-hooks ?
curious if anyone had any comments on https://cucumberbdd.slack.com/archives/C6QJ6N695/p1524678639000169
Wonder if anyone else is also experiencing a 404 not found error upon clicking on Cucumber tag expression here https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#tags
*Thread Reply:* Looks, like it moved to https://docs.cucumber.io/cucumber/api/#tag-expressions. Docs have been updated. Thanks for pointing this out!
*Thread Reply:* Yes. You can use Given When Then to define the step that uses And in your gherkin doc
This https://github.com/cucumber/cucumber-js/tree/master/features is great doco @charlierudolph. Doesn't look like it's linked to from the main README, but maybe it should be
*Thread Reply:* Yes. Didnt work So i installed 3.x and it would packaged the bin folder properly. Only difference between 3.x and 4.x in terms of bin folder is the change in name of the cucumber-js binary and addition of the run_slave binary in 4.x
@binarymist we’re considering basing all cucumber implementations on https://github.com/cucumber/cucumber-pickle-runner so any changes should probably go there.
*Thread Reply:* I use cucumber-html-reporter module and feed the data to it using the inbuilt json outputer.
*Thread Reply:* @arjunjh10 Yeah, somewhat that plugin is almost similar.
I've different metadata for different features - multiple-cucumber-html-reporter supports metadata to be read from the json(s).
My problem is how to get it in there?? 🙂
*Thread Reply:* @len Ok I’ll tell you the way I get the data through Json in case of my framework My Cucumber.js file
const targetResultJson = `./testoutput/<Filename>.json`;
const commonOptions = `--format json:${targetResultJson} --format summary --require-module ts-node/register --require ui/features/****/**.ts --fail-fast`;
my cucumber-runner.ts file
import ** as reporter from 'cucumber-html-reporter';
const outputDir = `./testoutput/${profileName}`;
process.on('exit', () => {
const options = {
brandTitle: 'Test Report',
theme: 'hierarchy',
jsonDir: outputDir,
output: `${outputDir}/${profileName}test-report.html`,
reportSuiteAsScenarsios: true,
launchReport: false,
storeScreenShots: true
};
// This generates HTML report.
reporter.generate(options);
});
Hope it helps
*Thread Reply:* ooh, nice. thanks @arjunjh10 , I'll try that process.on('exit' out soon.
Regarding my original problem. I forgot to mention, that we package the test project using nuget and upload it to TeamCity. On version 2.3.1 the bin folder was also packaged along with other Cucumber contents. It only seems to be excluding the bin folder from Cucumber 4.2.1. Also, I made no changes to my nuspec package creator file
So it sounds like the answer is: That there isn't currently a way to do this in the CLI, and if we want to implement something in Cucumber (so that the community benefits), then learn Go if we don't already know it, and start hacking on the cucumber-pickle-runner which isn't yet used by the CLI anyway? Alternatively just write some code to read the files ourselves and parse looking for the tags we want?
https://github.com/cucumber/cucumber-js/tree/3.x/bin
https://github.com/cucumber/cucumber-js/tree/master/bin
@JJ (or anyone else) - would you be interested in porting the new DataTable library to JavaScript? https://github.com/cucumber/cucumber/tree/master/datatable
*Thread Reply:* @Aslak Hellesøy just looked a little bit at it. Think its hard to make it work in a untyped language, as you don't specify the type of parameter in the step definition like with java
Is there a way to put tables into the step definition (or is this a good idea?) vs something you 'magically' get passed? ie. Given your feature file is:
When I add the following contacts:
| name | email | age |
| lennart | <a href="mailto:lennart@mail.com">lennart@mail.com</a> | 30 |
| grandma | | 80 |
Then the cuc expression in your step definition might be:
When('I add the following contacts:[contactlist]', function (contacts)
Instead of (currently)
When('I add the following contacts:', function (contacts)
? If we had this possibility - registering custom table types wouldn't need to be any different from other custom data types (e.g. just set the transformer to Contact.fromJson)?
Alternatively, we could pass a datatable type into the options of the step definitions (https://github.com/lenntt/cucumber-js/blob/master/src/support_code_library_builder/build_helpers.js#L54)?
*Thread Reply:* I like the suggestion to make it explicit in the expression @len! Angle brackets don’t mean anything yet, and they could be used to identify a table type (converter). We could also use |contactlist| (pipes) to make it look a bit more like a table. @mpkorstanje @charlierudolph I’d be interested in your thoughts on this.
*Thread Reply:* Braindump:
A strong type system is required. This means given a method
f(arg)we should not only be able to figure out the type ofarg, ifargis a list we should also be able to find it the list item type. (Currently Java8 Lambda expressions fail this test).If no type system is available we'll have to build a tiny expression language to describe the same type information. E.g.
"map of String to List of Double"or{String: [Double]}.I am currently thinking that passing this into the stepdef will be most worth while.
*Thread Reply:* E.g. Given("my expression", "list of strings") but I hold no strong opinions on this.
*Thread Reply:* 4 Expressing this type system can be done through a an expression in a string or through methods. E.g. "map of int to double" of map(Int, Double). Depending on what is available in the language.
*Thread Reply:* Owh. Better idea for js.
You we can declare the type as an object {"String": ["Double"]}.
*Thread Reply:* So to summarise - we provide a mechanism to declare the table transform type. This is for languages/syntaxes that don’t have static typing (Ruby, JavaScript) and/or sophisticated reflection (Java8 lambdas).
We haven’t settled on the mechanism, but the suggestions are:
* [...] at the end of a cucumber expression (I don’t really like that, thinking about it)
* a 2nd argument following the Cucumber Expression (or RegExp). The type/structure of this argument is still to be determined (String, Dictionary, other…)
*Thread Reply:* Dictionary --> Object. A list of double is ["Double"] a map of String to Double is {"String": "Double"}.
*Thread Reply:* @Aslak Hellesøy
1) square brackets are just an example, can live with other characters (I don't even see why not just {...}).
2) 2nd argument: there is already an optional 'options' object 2nd argument (https://github.com/cucumber/cucumber-js/search?q=buildStepDefinition&type=Code&utf8=%E2%9C%93).
I'm not seeing the 2nd work doing an automatic transform yet,
(my guess is most of the times, when writing my own transformer, I'll be calling a constructur or something - basic data types wouldn't suffice.)
@mpkorstanje, if we stick to the contactlist example above, what would the second argument look like?
*Thread Reply:* @len - Cucumber Expressions currently have no knowledge of DataTables. It is a standalone library. I think it’s cleaner to keep it dependency-free. That excludes declaring tables in Cucumber Expressions. It’s a bit weird to add it there - it has nothing to do with matching steps.
*Thread Reply:* @len It would look like this: When('I add the following contacts:', ['contact'], function (contacts))
And somewhere else you'd have to register a data table type:
typeregistry.defineDataTableType(new DataTableType(
"contact",
new TableEntryTransformer(entry => new Contact(entry));
));
*Thread Reply:* Think I got it, thanks.
So basically the only difference between the normal CustomDataType and the CustomTableDataType, would then be that the transformer in the latter iterates over all entries? @mpkorstanje
*Thread Reply:* That would depend on the transformer. You can iterate on the individual cells, the entries - (header, value) pairs -, the rows, or the whole table at once.
*Thread Reply:* That'd be TableCellTransformer, TableEntryTransformer, TableRowTransformer and TableTransformer respectively.
*Thread Reply:* These can be combined to some degree:
For example transforming this table to {String: Coordinate}
| | latt | long |
| KMSY | 29.993333 | -90.258056 |
| KSFO | 37.618889 | -122.375 |
| KSEA | 47.448889 | -122.309444 |
| KJFK | 40.639722 | -73.778889 |
Would use the build in TableCellTransformer for String and the custom TableEntryTranformer for Coordinate.
*Thread Reply:* If the table didn't have header you'd have to declare a TableRowTransformer instead.
*Thread Reply:* Ok, cool.
Seems option #1 (building into expressions) is not really an option? (I can still give some arguments for that)? @Aslak Hellesøy
*Thread Reply:* @len if you’re planning to port the library to JavaScript (would be amazing BTW), the right place to do that would be in a javascript directory under https://github.com/cucumber/cucumber/tree/master/datatable
*Thread Reply:* We can then work on integrating it into cucumber-js. How does that sound?
*Thread Reply:* Sounds great, I'll definitely look into it. If I have any, what would a good place to ask questions, this thread?
*Thread Reply:* Let's discuss it in the PR. Slack messages disappear after a while.
*Thread Reply:* Table + type --> json. Doesn't cover everything, but should cover the common parts.
*Thread Reply:* @Aslak Hellesøy @mpkorstanje PR is here, quite rough, but I need to know if I'm on the right track before continuing https://github.com/cucumber/cucumber/pull/385
*Thread Reply:* fyi you can wrap multi-line code snippets in triple-backtick ``` for easier reading
*Thread Reply:* like this jsontext = "[{"key":476007500100000067,"request time":"2018_05_03T12:47:39.115Z","response time":"2018_05_03T12:47:39.221Z"}]"
It attaches to the results.json as:
{
"key": 476007500200000100,
"request time": "2018-05-03T12:47:39.115Z",
"response time": "2018-05-03T12:47:39.221Z"
}
This is my go-to site https://yarnpkg.com/en/docs/cli/
*Thread Reply:* And, yes - you would simply write the step definition as the parent it belongs to.
For example:
Given I have a condition
And I have another condition
the And here would actually be a Given when you write out your step defs.
Iirc, But is no longer used.
*Thread Reply:* Yeah that's what I'm doing right now, replacing the Ands with Givens, but I was wondering if there was a way to add it so the step is And(/^bla bla$/)
thanks! 😄
*Thread Reply:* Perhaps you mean https://stackoverflow.com/questions/50104494/cucumberjs-and-gherkin-why-can-not-i-use-and-and-but-steps/50249795#50249795 ?
*Thread Reply:* @Yakim Rachev my first guess: async selenium js is async, so either your tests need a properly resolving promise or use a callback on the step definition
*Thread Reply:* Hi @len. Thaks for the quick reply. I'm aware of selenium async nature, and in my words.js file I call selenium functions with await.
*Thread Reply:* are you able to share a sanitized config file for your cuke and selenium setup?
*Thread Reply:* while i haven’t tried using async/await @Cat, i am having success just using promises.
*Thread Reply:* assertions not resolving promises were giving me grief. using chai-as-promised and asserting, say:
element.getText().should.become(text, 'The text did not match:');
*Thread Reply:* Hi there. I think I’ve got the problem… As I said I’m new to CucumberJS (also to NodeJS). In my step definitions was not returning the result from the webdriver methods call. I went to nightwatch-cucumber, now and the thinks happens more easy. Thanks all
*Thread Reply:* I get more failures when running on windows and I have to update the package.json to have "set BABEL_ENV=browser;
*Thread Reply:* In order to check my sanity I created a Docker container running ubuntu 16.04, npm 5.6.0, and nodejs 8.11.2, that is when I still get the linting errors and after the auto-fix I still get the one failing scenario.
*Thread Reply:* If I’m understanding the question correctly, you’d be able to simply pass the step defs folder into the require arg and just point to the features directory
*Thread Reply:* https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md you’ve followed this doc?
*Thread Reply:* '/nodemodules/cucumber/bin/cucumber.js --require ./nodemodules/sc-caf-module-test/test/features/step-definitions --tags @component ./tests -f json:./tests/reports/cucumber-report.json'
*Thread Reply:* Based on that doc I sent, it looks like your step defs should be passed through as : <DIR>/****/**.js
*Thread Reply:* although if you’re keeping it in the features dir it could just be: features/****/**.js
*Thread Reply:* ok, ill continue to play, as i have tried what it is saying in the doc, i guess im just missing something....
*Thread Reply:* np - out of curiosity, is your support code JS browser compatible? Or is it es6, etc?
*Thread Reply:* Bump of desperation. Does anyone have experience in world-parameters?
*Thread Reply:* I haven't used them, but what you could try is passing system environment variables instead?
TEST_ENV=production./node_modules/.bin/cucumberjs ...
(depends a bit on your setup and OS) then in your code:
if (process.env.TEST_ENV === 'production') ...
*Thread Reply:* @len Nice, that helped.
This is my package.json for those who may be facing similar problems.
"scripts": {
"test": "cucumber-js features",
"test-prod": "TEST_ENV=production npm run test",
"test-stage": "TEST_ENV=development npm run test"
},
then in my hooks.js
if (TEST_ENV === 'production') {
// Do prod setup
} else if (TEST_ENV === 'development') {
// Do dev setup
}
Are you using a custom world constructor? The world parameters get passed to it: https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/api_reference.md#setworldconstructorconstructor
@Alonso I don't think you can change the results the built-in json formatter writes back directly (if I'm wrong, please correct me!!),
but perhaps you can workaround with this: https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/attachments.md
I wouldn't mind a better solution to set, for a similar case, fill metadata property on a feature, e.g. to use https://github.com/wswebcreation/multiple-cucumber-html-reporter#metadata more directly.
*Thread Reply:* @mpkorstanje takes a while to get used to - I was in the cucumber-js docs like 'WHERE IS CUCUMBER-EXPRESSIONS' 🙂.
yeah, multiple-cucumber-html-reporter prints them, so do most others, like https://github.com/gkushang/cucumber-html-reporter
Sorry for newby question, but how do we turn on the cucumber-report.json in cucumber.js? I see this: https://github.com/john-doherty/selenium-cucumber-js/blob/7238c9bf966bab0ca0805445130c7754fb5e14aa/index.js#L100 but I'd like to get reports going on https://github.com/Adezandee/cucumber-mink
*Thread Reply:* @Vaene I dont understand the question, do you want want a json report output? or use cucumber-mink? (I dont think those two are related?)
I can probably only help with the first one. How do you run your tests now?
*Thread Reply:* Hi Len, thanks for the reply. Yes I just want to setup cucumber to write cucumberreport.json by default. I can do it via command line now (figured it out after I posted) with: npx cucumber-js --format json > ./reports/cucumberreport.json features/my-smoke.feature
*Thread Reply:* The selenium links is just by way of example of how they are handling it. I like everything about the selenium-cucumber, I just like the access to the mink step definitions that cucumber-mink has.
*Thread Reply:* Reminds me of web_steps, be aware, danger ahead: http://aslakhellesoy.com/post/11055981222/the-training-wheels-came-off
*Thread Reply:* Yeah I gotcha but the Gherkin appeal is also so non programmer stake holders can write simple tests in their tickets to see basic functionality replicated. 85% of what we do falls into this boring training wheel area, the rest we can handle with more specialized mocha, etc tests
*Thread Reply:* What I'd like to do is have the test run, output a cucumber_report.json, and then have cucumber-html-reporter render that in pretty html. the selenium repo does that automatically, I am trying to replicate that functionality in the mink repo.
The world parameters are an alternative to env variables. Docs on world: https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/world.md#world, features with examples: https://github.com/cucumber/cucumber-js/blob/master/features/world_parameters.feature
*Thread Reply:* While kotlin is interoperable w/Java and Android, the javascript channel probably isn’t the place to post? 🙂
*Thread Reply:* Just want to be sure that Cucumber will will work with all platforms for years to come
*Thread Reply:* We have more then 20 teams that stating using it and we need to know that in 1 year it will still work on all platforms and be supported, who can help me to understand this?
*Thread Reply:* @David Arshavsky we’re building a company around the open-source product. I’m one of the original OSS contributors, a co-founder of the company and the CEO. We don’t intend on going anywhere 🙂
*Thread Reply:* @David Arshavsky that said, we rely on contributions from the community to keep things going. We recently stopped supporting some Cucumber-JVM modules because we didn’t have bandwidth to support them: https://cucumber.io/blog/2018/05/19/cucumber-jvm-languages-support
*Thread Reply:* I suggest we move this conversation over to <#C5YHPPJMP|help-cucumber-jvm> since this doesn’t seem related to JavaScript/Cucumber.js
*Thread Reply:* As you can see from the stats in that post, Cucumber-Android accounts for 0.14% of the usage of Cucumber-JVM. Since you have 20 teams using Cucumber-Android, would you consider giving something back to the community by encouraging some of these developers to contribute to Cucumber-Android? That’s the best assurance you can get that it will stick around.
*Thread Reply:* The teams mostly using it for web and some for iOS, now we wanted to try it on Android as well, so I understand that you stop supporting because low numbers of usage at this point. I will speck with our QA leads and see what they say about supporting. Any numbers on iOS?
*Thread Reply:* Our intention is to continue to make new releases of Cucumber-Android ad the underlying Cucumber-JVM evolves. How long we’ll be able to do that depends on contributions from users. How many people use it without contributing doesn’t affect the decision as much.
*Thread Reply:* There is no official Cucumber for iOS, but there is a semi-official one that the Cucumber team doesn’t maintain. See http://docs.cucumber.io/installation/ and https://github.com/Ahmed-Ali/Cucumberish. It’s also in need of new maintainers. I don’t have any downloads stats for it.
*Thread Reply:* So I understand that your team focus at this moment are more for web then Android /iOS ?
*Thread Reply:* Our main focus is on 4 implementations: Java, JavaScript, Ruby and .NET. No Cucumber implementations know anything about the Web (HTTP,DOM etc). People can use any of the implementations to test web applications, but you need to add additional libraries to the mix for that (Selenium WebDriver being the most popular one). Cucumber doesn’t know or care what UI automation library you use to interact with a UI.
*Thread Reply:* ^^ to expand on that: There’s nothing stopping you from using appium/its equivalent in conjunction w/Cucumber. Same as any other runner.
*Thread Reply:* Unhandled rejection ReferenceError: Given is not defined
*Thread Reply:* How and where to define Given? 😞
Can you please just put a code here to open any web-page? (for e.g. google.com)
*Thread Reply:* ```_stepdefination.js__ import { Given } from 'cucumber'; this.Given(/^I am on the login page$/, function (callback) { browser.get("URL").then(function(){ callback(); // })
});```
*Thread Reply:* Still not working!
import { Given } from 'cucumber'; Given(/^I am on the login page$/, function (callback) { browser.get("https://www.mysite.com/").then(function(){ callback(); // })
});
*Thread Reply:* Many errors, I can see..
Unhandled rejection /home/ttpllt21/Desktop/cucumber/stepDefinitions.js:1 (function (exports, require, module, _filename, _dirname) { import { Given } from 'cucumber';
*Thread Reply:* node version / loader? try
const Given = require('cucumber').Given:
*Thread Reply:* Thank you all, but, still, nothing is working.
I am running it with Protractor
_conf.js_ exports.config = { directConnect: true, framework: 'custom', frameworkPath: '/home/ttpllt21/nodemodules/protractor-cucumber-framework', capabilities: { 'browserName': 'chrome' }, cucumberOpts: { format: "summary" }, specs: ['./**.feature'] };
--------home.feature--------- Feature: Homepage
Scenario: Visit Homepage Given I am on the homepage
------step.js------------
const Given = require('/home/ttpllt21/node_modules/cucumber').Given(/^I am on the login page$/, function (callback) { browser.get("https://www.mysite.com/").then(function(){ callback(); // })
});
*Thread Reply:* nothing is working is not enough for us to help debugging 😉
Did you turn on your pc? 😈
*Thread Reply:* Yes! PC is on! 🤣 script is not working, I am just new and have no basic idea about cucumber, I have tried to read articles and none of them helped.
*Thread Reply:* @Aasim Anwar I recommed you read this then ask again: https://opensource.com/life/16/10/how-ask-technical-questions
*Thread Reply:* the babel docs are pretty thorough as far as setup if you’re wanting to use ES6
*Thread Reply:* try to require step definitions exactly
require: './step_definitions/****/**_steps.js', => require: './step_definitions/login/login_steps.js',
*Thread Reply:* and it may have to be array, so require: [ './step_definitions/login/login_steps.js' ],
*Thread Reply:* It does not appear to be that as I've tried with or without wildcard**s and under array form.
*Thread Reply:* I remember having to change this.When(... to When(... (where when is required from cucumber)
*Thread Reply:* I'll try refactoring the steps when I have time, currently I've decided on 1.3.3
Hmm, thats very odd. I think that has to do with how the function is transpiled to javascript. Not much workaround aside from not using a default parameter. You could also use something like https://www.npmjs.com/package/util-arity to wrap your function in one that ha a specific length
Beside being a very interesting read https://eslint.org/blog/2018/07/postmortem-for-malicious-package-publishes Caused quite some noise at some offices this morning ^^
Be aware there's a timeslot you could've pulled this in locally when working on cucumber or other projects with eslint.
*Thread Reply:* What’s the issue precisely? are you unable to add the driver?
*Thread Reply:* thanks Cat for your response. I was facing issue while running test on IE11 . Issue has come up after adding driver as it was asking to add gecho.
Issue is resolved now. Thanks a lot:)
I’ve been using this framework for testing some applications that use Angular as well as legacy applications. https://github.com/igniteram/protractor-cucumber-typescript
<a href="https://github.com/igniteram/protractor-cucumber-typescript">igniteram/protractor-cucumber-typescript</a>
I created issue about it https://github.com/cucumber/cucumber-js/issues/1117
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I am not able to get it working...setting tags in the hooks is same as defining a tag. needs to be invoked by explicitly declaring the tag in the feature file.
*Thread Reply:* After({ tags: 'not @deleteParticipant' },(scenario){//do something}
*Thread Reply:* I have few After hooks with tags, as such
Before({tags:'@setup', async(){
//does something})
After({ tags: '@teardown' }, async () => {
//does something}
which is called in a feature file as such
@setup @teardown
Scenario: Do something
I also have an after hook with no tags, which runs after each scenario
After((scenario){//take screenshot}
I need to take a screenshot when a step fails in the scenario.
*Thread Reply:* The above works great as long as i dont have the @teardown tag in the scenario. The screenshot is taken at the step it fails.
*Thread Reply:* For scenarios that have the @teardown tag, the screenshot is taken at the end of tagged after hook, @teardown
*Thread Reply:* What i am trying to do is to see if i can take the screenshot at the time of step failure, instead of scenario failure.
*Thread Reply:* Ah, gotcha. I'm not sure the best way to navigate that. I do know that the scenario object that gets passed in to the hooks contains the info around what tags it has as well as its success/failure, so perhaps you could modify the hooks to instead perform actions based on particular tags?
*Thread Reply:* now that i think about it..it might not work. The After hook with no tag runs after the complete scenario, including @teardown. I might be wrong, but will let you know 🙂
*Thread Reply:* You could just have your scenario.failed? check happen before teardown and theoretically be golden. 🙂
*Thread Reply:* Thanks so much @Gem! i was able to find a solution using your suggestion.
*Thread Reply:* The first line of that link is literally “World is an isolated context for each scenario, exposed to the hooks and steps as this. ”
*Thread Reply:* @Cat Hello again, is there a way I can tell cucumber not to process.exit
*Thread Reply:* iirc, cucumber exits by default when the event loop has finished draining. I’ve not had a need to do this, so I’d suggest looking at the CLI code for this one
*Thread Reply:* Oh sorry I just had bug in the output 😥 it actually doesn't exit by itself. Thank you 😄
*Thread Reply:* (don’t laugh - I legit just used it this morning already for some weirdness I was seeing - turned out to be a Babel issue)
*Thread Reply:* Hi again, I'm using this and it's working great with all my requirements except that when I try to pass the --parallel option it gives me an error
*Thread Reply:* do you think it should work if we run the cucumber.Cli like this
*Thread Reply:* of course I checked that I don't have any writes to the console
*Thread Reply:* keep in mind --parallel is experimental (if I recall correctly - one of the others may correct me on that)
*Thread Reply:* Cypress is great! I use this (below) tool to combine these two amazing tools! https://github.com/TheBrainFamily/cypress-cucumber-preprocessor
<a href="https://github.com/TheBrainFamily/cypress-cucumber-preprocessor">TheBrainFamily/cypress-cucumber-preprocessor</a>
*Thread Reply:* I’m not able to dig into the stacktrace right now, but I’m wondering if the fact that you’re returning cucumberCli.run() may be causing some issues (I could be wrong - just a hunch based on glancing at the code)
*Thread Reply:* it returns a promise so I need to listen to it to know when the tests have ended
See these docs for the latest way to attach: https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/attachments.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
Hi 👋 I am using cucumber v4. An exception thrown in BeforeAll hook crashes the whole run. Please advise, is there any event/handler available for this case? So if an exception takes place, I want to be notified of that, close other processes (configuration server, connection to remote report dashboard, etc.) and only after this to stop the run. Found similar issue (in relation to afterStep), though it was 2 years - https://github.com/cucumber/cucumber-js/issues/658. Guess, things have changed since that time. Any advise will be highly appreciated 🙂
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Can you give an example of what you’re doing in particular? I don’t think it should need to be in a particular position
*Thread Reply:* i am passing through additional command line variables to allow configuration by TeamCity of various test configurations
*Thread Reply:* afaik, there’s no reason it would need to be passed in a particular order, or drop the params.
*Thread Reply:* but a custom value — say --target dev is swallowed somewhere when --parallel 25 or whatever is provided
*Thread Reply:* it only seems to be the ’--target` option. others pass through fine.
*Thread Reply:* Integration with Testcafe...I building off this project https://github.com/rquellh/testcafe-cucumber but I'd ideally like to move the setup to Testcafe to BeforeAll rather than Before stage
<a href="https://github.com/rquellh/testcafe-cucumber">rquellh/testcafe-cucumber</a>
*Thread Reply:* Reason for moving to that stage is because setting up at Before for every test scenario is really slow
*Thread Reply:* So prior to invoking test cafe I'd like to generate the same number of testcafe scripts as is in the current test run with cucumberjs
*Thread Reply:* I think I’ve got you (although haven’t worked w/testcafe so bear with me). I’m not sure if you could actually grab a list of scenarios going into the run you’re executing since BeforeAll() doesn’t have the same context readily available as Before() but I don’t see any reason you couldn’t set up a fairly straightforward file reader to do the trick.
*Thread Reply:* I guess I could even use the gherkin library shipped with cucumber to get all the tests in a folder...it gets complicated tho if figuring out what tags were applied to the runner too & what tests not to generate... appreciate any pointers on the above
*Thread Reply:* hm. Since you can’t access world context from there, perhaps something that captures the args you use at CL and filters the file matching that way?
*Thread Reply:* I guess I could put the CL arguments into an environment variable & use it back at that stage...it does seem cumbersome tho 😔
*Thread Reply:* a bit. There may be a more clever way to implement it but that’s significantly different than how I’ve been using the BeforeAll + AfterAll hooks so I don’t want to give some super-simple sounding answer if it’s not remotely helpful 😂
*Thread Reply:* In any case it's defo not possible to get that info from BeforeAll...I know this isn't a typical use case but it's just to have integration with Testcafe
*Thread Reply:* I notice this.result has some info within BeforeAll but doesn't return me all the feature files in that run
*Thread Reply:* it’s not impossible to get it. It’s just a bit of heavy lifting 🙂
*Thread Reply:* But by default from the framework cucumberjs it's a no!?
*Thread Reply:* Correct. BeforeAll & AfterAll aren’t designed to have access to the world instance because they run before everything.
*Thread Reply:* @Cat Hi Cat, I just saw this reply after logging in after some time. Thank you for replying, although I got a co-worker's help to figure this out. But again, thanks for volunteering to help
*Thread Reply:* Cucumber doesn’t capture screenshots. Your webdriver does that.
*Thread Reply:* If you want a screencap at every step, you could just wrap it so that screencap happens after promise is fulfilled by the test step
*Thread Reply:* some discussion here: https://github.com/cucumber/cucumber-js/issues/997]
*Thread Reply:* ^^ correct. That’s why I was thinking perhaps wrapping the steps themselves (perhaps at defineStep?) as promises and capturing the screenshot when they resolve would do
*Thread Reply:* Sometimes I know things, bringing it to a total of like 8 things I know! 😄
*Thread Reply:* Currently sniffing at https://www.npmjs.com/package/ember-cli-yadda
*Thread Reply:* I’m also writing Ruby, and tests run there, as well, so it could be something along these lines: http://trianglegrrl.github.io/ember-cucumber-pres/#/
*Thread Reply:* I’ve got nothing much to add, but just wanted to chime in that we use Ember and dig Cucumber, too (though we don’t use Cucumber with Ember at the moment)
*Thread Reply:* How it works on a Create React App is going to be the same way it works on any other app. Cucumber doesn’t care about your framework 🤷 What is the exact problem you’re trying to solve?
*Thread Reply:* The problem my team is facing is using react components with the stepdefs.js file. When we run the cucumber tests, we get an unexpected identifier at “import React, { Component } from ‘react’”;
*Thread Reply:* This import is located on the React component, we are testing. (Apologies for the lack of information, on my phone).
*Thread Reply:* ok, when you’re on your machine, ping me and send some of the code (sanitize it if you need to) for how you’re attempting to initialize the tests and access the app
*Thread Reply:* Ping! The issue we were having is we weren't transpiling ES6 Imports and React JSX. The issue was solved by using the following NPM packages: @babel/core, @babel/cli, @babel/preset-env, and @babel/preset-react. We are still having issues using React CSS imports (Node doesn't like the CSS syntax), ~but taking out those imports allow us to test for now~. EDIT: We got CSS to work too! For every component under test, you have to check if the component is being used in the browser or the server. If it is being run in the browser, "require" the CSS files. It if is being run in the server. do not "require" the CSS files.
*Thread Reply:* hm. You should be able to get around that by leveraging babel runtime or modifying where you’re introducing babel to the project. That would enable you to keep the code consistent throughout
*Thread Reply:* Hi @Andy Nguyen I have the same problem and I've install the packages that you mentioned, but still no result. Still getting the "import React from 'react'" error. I've tried to import all things with require and it works, but then it doesn't recognize the JSX syntax, because I'm using shallow from enzyme. Do you have any idea how I can solve this problem. Thanks
*Thread Reply:* We are also using enzyme, so that shouldn’t be the issue.
*Thread Reply:* Can you show me some example, or article how to setup all together and use it, I will be very thankful
*Thread Reply:* I’d just look at the babel.io docs. They’re pretty thorough re: setup and config docs.
*Thread Reply:* @TVlasev I made a tutorial and example project of how to configure Cucumber, Enzyme, and Create React App. Hopefully having this example project and tutorial will encourage more people to use Cucumber with React. Let me know, if this helps you https://github.com/NguyenAndrew/Enzyme-Cucumber-React
<a href="https://github.com/NguyenAndrew/Enzyme-Cucumber-React">NguyenAndrew/Enzyme-Cucumber-React</a>
*Thread Reply:* @Andy Nguyen Great repository, looks really good, I'll try it out and let you know if helps, thank you for the effort
*Thread Reply:* @Andy Nguyen It's working, it's great repository, I'll spread the word, hope more people will use it, thank you! Just one more question: If we have a normal tests with jest and enzyme and we want to run them too, i guess i can just add "react-scripts test", like that: "test": " react-scripts test && npm run babel-src && npm run babel-test && npm run cucumber-test" ?
*Thread Reply:* @TVlasev Yes, that should work as the create react app was not ejected.
*Thread Reply:* @Andy Nguyen Just like suggestion in FAQ / Troubleshooting section you can include what to do if css files throws an error and how and where to include them.
*Thread Reply:* @TVlasev Ah thank you for the suggestion. I am hesitant on adding the Jest react-scripts question to the FAQ, because it is more of a “How do I use npm?” question rather than “How can I integrate Cucumber, Enzyme, and Create React App?”. Another reason: Cucumber plays very with many different testing frameworks, so I am trying to keep the tutorial focused. Maybe I will make an other questions page?
*Thread Reply:* @Andy Nguyen maybe i did't express myself well, sry for the bad english 😄 If we use a css file in a component when we run npm test, it gives this error: Unexpected token . Copy\transpiled\style\style.css:1 (function (exports, require, module, _filename, _dirname) { .search-bar { I think it's related to the subject, because if we use a imported css file in the component we can't test the component because the error. Node doesn't understand css syntax. Correct me here if I'm wrong
*Thread Reply:* @TVlasev That question is in the FAQ of the repo: "I have a Syntax error: (function (exports, require, module, _filename, _dirname) { .App {
SyntaxError: Unexpected token ."
*Thread Reply:* You should be able to ctrl+f that question, as example code is provided below the answer to solve that issue. Let me know, if you still have problems.
*Thread Reply:* @Andy Nguyen ohhh, I'm sorry, i saw it. But still gives me the error when I use a css selector on JSX element. In src/ folder I have a style/ folder with style.css file in it. And I do the import with "import: ../../transpiled/style/style.css" in stepdefs.js file but still gives the same error.
*Thread Reply:* @TVlasev I am confused on this part: "And I do the import with "import: ../../transpiled/style/style.css" in stepdefs.js file" Can you clarify on what functionality you are testing when you import the CSS into your stepdefs? From my experience, you import/require the CSS into the corresponding react component (checking if the dom exists like you read in the FAQ), and the stepdefs.js would import the react component,
*Thread Reply:* @Andy Nguyen Now i get it.. I've deleted the canUseDom func... 😄 now everything is ok.. sorry for this. Thank you again for your time
*Thread Reply:* @TVlasev 😄 Alright. Interesting that deleting the canUseDom functionality solved your problem. You are welcome
*Thread Reply:* @Andy Nguyen no, no I've deleted it by mistake, after I added it back everything was OK 😄 Sry again for the english 😄
*Thread Reply:* Cat's advice led me to an awesome discovery, which should be beneficial to all React Cucumber developers (Unit Tests with no drawbacks, no rearranging of the step_definitions folder, and no transpiled folders). ~I opened an enhancement ticket on the tutorial project~ Enhancements have been implemented!
<a href="https://github.com/NguyenAndrew/Enzyme-Cucumber-React">NguyenAndrew/Enzyme-Cucumber-React</a>
*Thread Reply:* @TVlasev I have implemented major improvements in the tutorial that improves the developer experience when using Cucumber with React. I would recommend updating your project to use them.
*Thread Reply:* I don’t have any samples off hand. what in particular are you having trouble with? the cucumber setup docs are pretty straightforward 🙂
*Thread Reply:* I’m mid-meeting so can’t look it up rn, but I believe it’s just Before and BeforeEach
*Thread Reply:* Thank you for taking time to reply. 🙂
is there a better way/strategy to do a "setup" before any scenario runs?
*Thread Reply:* Hm. I would say it depends on precisely what you’re needing to do.
*Thread Reply:* @Cat so dumb of me..I put all my steps for data setup in before hook. This runs before any of the scenarios runs. not sure if this is the most efficient.
I am not sure if it runs before every single scenario
*Thread Reply:* I do know if its a tagged hook, it will only run when called in a feature file. a general Before hook i am guessing will run before every scenario. Testing this theory..
*Thread Reply:* hm. So, ymmv because how one automates one piece of software may vary wildly from the next, but for me, I find it helps to do data setup as part of the specific feature or scenario - usually in the Background steps. Now, there are some things that are more generic (for example: if you want to log in via API and get cookies - that’s likely something that applies to a ton of your tests and you can just do that as a before hook and pass the cookie values through there)
*Thread Reply:* But again, that is going to vary greatly depending on the how and why 🙂
*Thread Reply:* there are definitely cases where handling the data setup is completely legit in the Before hook
*Thread Reply:* or it may be that some of the data setup should be in the Before hook, but other pieces need to be done only before tests with specific tags, etc
*Thread Reply:* That's exactly what i intended to do with the data setup(maybe data setup is not the right word here). There are some reference data, that are constant that are needed as a pre-req, i am using API calls to prepopulate them. Ther are also other data setup that are specific to a feature file which are done via background. And lastly there are ones which need some, random data to be created, these are done with tagged hooks.
Hello people 🙂 I have a question regarding cucumber.js. I hope this is the right place to ask it 😛
So, Is it possible to get currently executed test tag in beforeScenario function? If it is possible how to do it? 😄
Here is the link on stackoverflow where I asked the question https://stackoverflow.com/questions/53729101/cucumber-how-to-get-scenario-tag-that-is-currently-being-executed
but with no luck yet for answers 😁
*Thread Reply:* Hm, are you using straight cucumber-js or are you using wdio-cucumber?
*Thread Reply:* Hm. that’s quite an old version. 🤔 That may be part of why you’re having difficulty. I want to say a few things have changed about how tags are fed in as far as where they’re available to reference, so I wouldn’t easily be able to answer that straight off
*Thread Reply:* tbh, I’d start first with just playing around with console.log to see if you can just get it to log the tags and go from there.
*Thread Reply:* yeah, it's like that on the project I'm currently working on and upgrading the version is not an option at the moment 😞
*Thread Reply:* I can get the scenario tags by this:
beforeScenario: function (scenario) {
tags = scenario.getTags();
tags.forEach(function (scenarioTagItem) { ... });
}
*Thread Reply:* but what I can't figure out is how to address/catch the tag of the test that is currently being under execution.
*Thread Reply:* Hm. That I probably can’t help much with until I have some time to play with a test project
*Thread Reply:* So for example, I have scenario with multiple tags, but conf.js file only running certain tag @registration and I want to catch that this scenario is being run under that tag => I know what are all tags which with scenario is tagged, but how do I catch the active one
*Thread Reply:* Hm. Since it’s an older version and I’m just guessing here, I don’t want to point you in the wrong direction, but what happens when you try to reference something along the lines of this.scenario? Are you able to access tags that way?
*Thread Reply:* when I do <a href="http://logger.info">logger.info</a>("XXX scenario " + scenario.toString()); I get info: XXX scenario [object Object]
*Thread Reply:* when I do <a href="http://logger.info">logger.info</a>("XXX scenario " + JSON.stringify(scenario)); I get info: XXX scenario {}
*Thread Reply:* I have in conf.js file this among some other things:
*Thread Reply:* before: function () { const chai = require('chai'); global.expect = chai.expect; global.should = chai.should(); global.logger = require('winston'); global.i18n = i18n;
browser.setViewportSize({width: 1200, height: 1057})
}
*Thread Reply:* so I guess it is using winston for logging purposes
*Thread Reply:* yes, sounds like - try just console.log instead and lmk what you get
*Thread Reply:* console.log("XXX scenario " + JSON.stringify(scenario)); => Still the same XXX scenario {} 😞
*Thread Reply:* Hm. that is odd. I’m afraid I’m not gonna be much help on this until I have some time to poke around with that version
*Thread Reply:* getting stuck with older versions is always difficult 🙂 to attempt to point you in the right direction - I’d keep playing around with that object and see if there’s anyway to successfully destructure it and get more info on what’s (supposed to be) in it
*Thread Reply:* I managed to debug it and searched with offered options that were there, but I think none of it will help me
*Thread Reply:* What I really need is a a way to get the tags from configuration file main.conf.js that are set in the attribute:
cucumberOpts: {
tags: ['@registration', '~@WIP']
}
in function beforeScenario?
*Thread Reply:* not the ones that are actually on the scenario itself
*Thread Reply:* and all of these are actually getting those tag that are on actual scenarios
*Thread Reply:* I did it :nyanparrot: it was stupid => just called this.cucumberOpts.tags because it was in the same file and build my logic further on that
*Thread Reply:* but thank you again for you time and help, I really appreciate it 🤗
*Thread Reply:* Just curious - what’s the need for retrieving tags at run-time ?
*Thread Reply:* I have multiple profiles in wdio for different purposes (in this specific case countries) and I needed logic to know for which country did tests run @fr , @uk and so on @Andrew Peacock
Hi everyone, is anyone here able to review a small JavaScript example to add to the docs? https://github.com/cucumber/docs.cucumber.io/pull/287
<a href="https://github.com/cucumber/docs.cucumber.io">cucumber/docs.cucumber.io</a>
*Thread Reply:* I am not a "reviewer", but I did take a look and made some comments.
*Thread Reply:* Please stop by the docs anytime to improve the javascript side of things ;)
*Thread Reply:* ```export default async function run () { const cucumberCli = new cucumber.Cli({ argv: execArgs, cwd: process.cwd(), stdout: process.stdout });
var cliRun = await cucumberCli.run();
process.exit(cliRun.success ? 0 : 1);
}```
*Thread Reply:* Since the junit formatter requires piping the output the exit code being returned to Circle was for formatter which always passed, so Circle thought the Cukes were always passing, too
I have not solved this problem before, but maybe this will help @Gem https://stackoverflow.com/questions/1221833/pipe-output-and-capture-exit-status-in-bash
*Thread Reply:* I couldn't get it working and didn't want did want to bash my head against it so decided to switch to Ruby. 😛
First, look at this page to see how to use promises with Cucumber.js https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/hooks.md. Notice that instead of doing async/await, you return the promise you want to be resolved. This guarantees the promise has resolved in each step, before moving onto the next step. If you have multiple promises you can return Promise.all([promise1, promise2, ...]) https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
Also, I currently have X groups of tests, and for each group I would like to create a custom World object. Do you know how to achieve this? More specifically, I would like to be able to define an interface in TypeScript for the variables that would be stored in each world object. I saw this article https://sylvain.pontoreau.com/2018/04/30/typescript-cucumber-getting-started/, which describes how to add types to the default World object (so you are able to get intellisense suggestions) but I was wondering if this is possible for custom Worlds.
*Thread Reply:* @Andrew From my understanding Cucumber.js doesn't allow you to share state between different scenarios (and may be an anti-pattern? I need confirmation on this statement). You can use the before/after hooks with tagging to setup and tear down state for each scenario in a group https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/hooks.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* @Andrew I have raised this issue which would be a great feature that could allow the configuration of multiple world classes based on test type. Feel free to chime in what any other suggestions. Another way you could do it is by having different support folders declared in your cucumber profile but I'd like to stay away from separation by folder and instead let tags drive world configuration. https://github.com/cucumber/cucumber-js/issues/1174
*Thread Reply:* How are you passing it into Cucumber? (cli or cuke options - just need to see how you’re passing it in)
*Thread Reply:* Hi cat, I used the ./node_modules/.bin/cucumber-js command
*Thread Reply:* all works well when the features folder is in the root directory, but I can't seem to find any way to run the tests if the folder is elsewhere
*Thread Reply:* Right. Cucumber has to be told where to read files from (if they aren’t in default locations). How are you passing that in?
*Thread Reply:* Not sure where i'd be passing that in. Sorry I'm brand new to cucumber
*Thread Reply:* I did try cucumber.js tests/cucumber/features/example.feature as well as --require but neither worked
*Thread Reply:* Try node_modules/cucumber/bin/cucumber-js test/features You shouldn't need require
*Thread Reply:* I made a full stack Cucumber React boilerplate last night with a feature folders in a test directory (instead of root). If the previous advice doesn't work, taking a look at this repo should help you run feature folders in directories outside of test, https://github.com/NguyenAndrew/careen.
<a href="https://github.com/NguyenAndrew/careen">NguyenAndrew/careen</a>
*Thread Reply:* Hi, it's cukebot again. "Doesn't work" is rather vague. What are you seeing and what had you expected to see?
*Thread Reply:* Thanks, Andy. I did try that and i'm getting 'no such file or directory'
*Thread Reply:* I restarted my terminal, and now i'm no longer getting 'no such file or directory' but it thinks my feature is empty:
0 scenarios
0 steps
0m00.000s
*Thread Reply:* In your folder structure, can you indicate for me where your feature files and step definition files are located?
*Thread Reply:* Sure. here's my folder structure right now:
project
|-- tests
|-- cucumber
|-- features
| --example.feature
| --step_definitions
|-- stepdefs.js
*Thread Reply:* So instead of doing node_modules/cucumber/bin/cucumber-js test/features Try doing the following node_modules/cucumber/bin/cucumber-js tests/cucumber/features
*Thread Reply:* that worked perfectly. thank you both for your help I really appreciate it
*Thread Reply:* btw Andy that boilerplate is awesome! i miss using react 😢
*Thread Reply:* Thanks for the compliments. Hope you can jump back to the React world soon.
*Thread Reply:* You can set the timeout globally with: ```var {setDefaultTimeout} = require('cucumber');
setDefaultTimeout(60 ** 1000);``` This link show you how to set the timeout both locally and globally https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/timeouts.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Thanks Andy, I’m using protractor.conf.js to run my tests.
I’m not sure where this will fit in this file. I get an error when I set the timeout in it.
var {setDefaultTimeout} = require('cucumber');
setDefaultTimeout(60 ** 1000);
error:
setDefaultTimeout(60 ** 1000);
^^
SyntaxError: Unexpected number
*Thread Reply:* iirc (been a while since i used protractor), you can just pass it into cucumberOptions in the conf
*Thread Reply:* This is what I currently have:
cucumberOpts: {
require: ["features/step_definitions/**", "support/hooks.js"],
format: "json:./reports/cucumber_report.json",
tags: ['~@todo', '@regression'],
},
what it the proper way to add setDefaultTimeout(60 ** 1000) to it?
*Thread Reply:* hm. try plugging in defaultTimeout: (yourTimeout) or just timeout: (yourTimeout) in that config.
*Thread Reply:* I handle the setDefaultTimeout bit in my world setup code so how I pass it in is a bit different than what you’re doing. However, iirc, it should be passed in through options.timeout if it’s coming through as cucumberOpts
*Thread Reply:* I tried plugging in defaultTimeout and also timeout but none worked.
*Thread Reply:* hm. I’d suggest having a look at the protractor-cucumber-framework code to see if it exposes cucumber timeouts and if perhaps it has a different naming convention?
*Thread Reply:* I created a new file env.js and added
module.exports = function() {
this.setDefaultTimeout(10 ** 1000);
};
then added this to protractor.conf.js under cucumberOpts
require: ["features/step_definitions/**", "support/hooks.js", "support/env.js"],
*Thread Reply:* There has to be a simpler solution but this works for now. Thanks @Cat @Andy Nguyen!
*Thread Reply:* @Omar In your protractor config file, there is an onPrepare method available to you. You should be able to set the timeout in there.
*Thread Reply:* Let me know if it doesn't work and I can dig a little more but i suspect it will
*Thread Reply:* Hi, it's cukebot again. "Doesn't work" is rather vague. What are you seeing and what had you expected to see?
*Thread Reply:* This seems more like a selenium webdriver question than a cucumber question. However, I did look into this question for you. https://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/ie_exports_Driver.html#getPageSource returns a promise to a string. If you don't know promises, callbacks, or how asynchronous code works in JavaScript definitely do some research before continuing past this code. In any case do driver.getPageSource().then((source) => { if source.contains("Text to check") {...} else {...} });
*Thread Reply:* Thank you so much @Andy Nguyen!. That worked with a minor change. .contains does not work in chrome. So, I had to use source.indexof https://stackoverflow.com/questions/19589465/why-javascript-contains-property-is-not-working-in-chrome-browser. Anyways, thanks again 🙂
*Thread Reply:* I agree with @Cat a custom timeout on each call is a bad choice. My opinion is to increase the cucumber timeout.
*Thread Reply:* The problem isn't find the element. I want to handle errors when it happens. I'm trying to nominate my errors to do a better log for the UI Tests.
*Thread Reply:* Agreed the problem isn't finding the element because you actually don't know if it's being found or not because cucumber keeps timing out first.
*Thread Reply:* Yes. My tests are working fine. But I need to do a error handling because I want to make the code more maintable for everyone.
*Thread Reply:* ^^ Raymond is correct. The issue is that you’re trying to solve it in the wrong place - you need to give cucumber a longer timespan than the underlying driver functions in your step definition. If you run a stepdef with cucumber and set timeout to 2k, but your driver is set to 3k, cucumber gets the error. Cucumber has already failed your test before driver can finish attempting to resolve - this is exacerbated further if your stepDef is a helper function that waits for multiple driver functions to resolve.
*Thread Reply:* Yes. And you cannot capture an error in your driver if you aren’t waiting for it to fail
*Thread Reply:* You fail in cucumber right now before giving driver time to fail
*Thread Reply:* You will not capture an error at driver level if you consistently timeout in cucumber (because that wraps your driver step)
*Thread Reply:* const selector = '#login-botao1';
const promise = new Promise((resolve, reject) => {
setTimeout(async () => {
await this.document.waitForSelector(selector);
resolve();
}, 20000);
reject();
});
try {
await promise;
} catch (error) {
throw new Error('Unable to find element');
}
*Thread Reply:* I change the code for that and receive this message: Error: Unable to find element
*Thread Reply:* But the problem is that always got reject...
*Thread Reply:* Looks to me like the same problem with the only difference being you now have control of the problem
*Thread Reply:* Have you confirmed waitForSelector works. Also what error is being received into the catch? I presume the timeout error again right?
*Thread Reply:* × failed Error: Unable to find element at Login.<anonymous> (\tests\acceptance\pages\login.ts:33:15) at step (\nodemodules\tslib\tslib.js:133:27) at Object.throw (\nodemodules\tslib\tslib.js:114:57) at rejected (\node_modules\tslib\tslib.js:105:69)
*Thread Reply:* I would like to have this error on my log and I got this
*Thread Reply:* Can you console log the error the catch is getting before you throw your custom error
*Thread Reply:* Have you confirmed your waitForSelector method actually works?
*Thread Reply:* Okay can you console print the error entering in the catch
*Thread Reply:* That doesn't sound right You did console.log(error) at the start of the catch right?
*Thread Reply:* I resolved the problem using promise (resolve, reject). I remove some calls in the function and worked fine. Thanks for the help! 😁
*Thread Reply:* Hi! "Guys" is easily replaced with the more gender-neutral "Folks", "All" or "Foolish mortals". We'd appreciate if you tried to use that. Thanks!
*Thread Reply:* That’s going to vary quite a bit based on which driver you’re using. It’s also not really a cuke question 😉
*Thread Reply:* Like @Cat says we are going to need some more context. Also that is a selenium question and can be asked in #selenium
*Thread Reply:* If I’m not misremembering, you can just pass in your browserstack config via your webdriver config (webdriverio has some good examples). It’s going to vary some based on what you’re using, but isn’t so much a cucumber question.
*Thread Reply:* The Browsertstack repo uses the selenium webdriver and I guess the hooks.js file to configure it to use browserstack. But it only works with the old versions of cucumber and selenium the repo uses. So you think I can easily build it myself with new versions? Do you maybe have a good video or documentation on that
*Thread Reply:* webdriverio is a wrapper for selenium webdriver + appium. whether you’re using selenium webdriver or webdriverio, you should be able to pass the Browserstack parameters through the config. BrowserStack has docs here: https://www.browserstack.com/automate/node
*Thread Reply:* It’s an issue with how you’re interacting with the DOM, hence the selector error. It’s pretty much entirely unrelated to cucumber. 🤷♀️:skintone2: The selector you’re using cant interact with the dom in the way you’re attempting to. The most likely cause is that you’re attempting to set the input value in the wrong place. If you’re still struggling with it, you may want to check out #selenium or seleniumhq.slack
*Thread Reply:* Can you open up the divs as it looks like First name is a header and not an input field
*Thread Reply:* Hi @Gem: yeah, the problem is because there is no input tag. I will see if there are some workarounds. thank you.
*Thread Reply:* I can try put together an example this weekend if I get time but the cucumberjs docs should be all you need https://github.com/cucumber/cucumber-js/tree/master/docs
*Thread Reply:* So at a lower level chai is the runner? https://github.com/cucumber/cucumber-js/blob/master/docs/nodejs_example.md
*Thread Reply:* Also when your using it a a lower level what are you using mocking?
*Thread Reply:* No but it doesn't matter. Cucumber has no dependency on what it is testing. Cucumber is simply a framework to trigger some code to run
*Thread Reply:* I haven’t worked with that particular implementation, but I’d assume that you can probably just use selenium standalone, which can get your grid sorted out. https://www.npmjs.com/package/selenium-standalone
*Thread Reply:* https://cucumberbdd.slack.com/archives/C60TKS3SL/p1551200783073000?thread_ts=1551164882.031900&cid=C60TKS3SL
*Thread Reply:* Ah I see @Raymond, if you don't mind would you happen to have an example I can maybe look at?
*Thread Reply:* @Leo
- Add the formatter for reurn to the command line when running your tests(or add it to the profile)
--format rerun:@rerun.txt - After a failed run, remove any arguments used for selecting feature files and add the rerun file in order to rerun just failed scenarios. The rerun file must start with an @ sign in order for cucumber to parse it as a rerun file instead of a feature file.
*Thread Reply:* @Alexandros Katechis In the directory mentioned in the this.attach method or by viewing the report
*Thread Reply:* You’d need to leverage hooks and/or stepDefinitions. We’d need more information to be able to help beyond that.
*Thread Reply:* in my experience with cucumber, it’s easier to iron out expected behavior with the entire team - designers (UI/UX)/product, quality, ops, automation, frontend, and backend engineers. Also scales much more easily for growing companies. Mocha is fine for unit tests, but isn’t precisely a BDD framework (in spite of the fact that it lists itself as such). I find it’s perfectly useful for unit, but outside of that, I think it tends not to lend itself to scalable infrastructure like cucumber can.
*Thread Reply:* Mocha is very much like RSpec and Cucumber grew out of the RSpec project and it was always meant to be complimentary.
*Thread Reply:* I see, so cucumber is better for end to end integration tests?
*Thread Reply:* Yes. Particularly if you need to be able to show them externally
*Thread Reply:* I’d much rather walk through cucumber with my PO than mocha 🙂
*Thread Reply:* I’d say that Cucumber is better for customers tests (business facing) and mocha is better for programmer tests (technical facing).
*Thread Reply:* https://www.youtube.com/watch?v=XgVwU-HmC_A
*Thread Reply:* I think it varies. There are companies I’ve worked for where Cucumber for devs would have been a huge help. There are other companies where it would have been overkill. Depends so much on the subject matter + complexity of stack + how many distinct teams are working on multiple projects that interact, etc.
*Thread Reply:* My last job was entirely backend.. Not the usual use-case for Cucumber. However, the backend was a knowledge expert engine. Being able to essentially build a DSL for rule conditions -> expected outcomes was surprisingly helpful.
*Thread Reply:* My current team has pretty much only back end services and we use Cucumber. The services contain business logic (i.e. behaviour)
*Thread Reply:* When I say Customer Test and Programmer Test its not about the level of the stack we’re talking about. It’s more about who “owns” the test, who’s responsible for devising the parameters of the test and who really cares about the outcome.
Customer Test could just as easily be Business Test – for me it’s really about whether it describes the business process. It’s for describing what the software does in a way the business can understand.
A programmer test is to help the programmer write the software. It might give confidence of the correctness of the code, it might serve as an example of how to use a function or class, or if your lucky it can give you fast feedback on your design.
I’d expect most projects to have both customer and programmer tests 🙂
Cucumber is excellent for customer tests.
Lots of great stuff here on the c2 wiki: http://wiki.c2.com/?CustomerTest http://wiki.c2.com/?ProgrammerTest
*Thread Reply:* Yes, I understand the difference between programmer and customer tests, and that they're not tied to the level of the stack. E.g. in my previous team we had unit tests on front end logic. I'm just saying that Cucumber can be used at different levels of the stack. I've used it on front end and back end.
*Thread Reply:* A misconception I notice is that Cucumber is for front end (only)
*Thread Reply:* how can i do what using the webdriverIO framework with wdio test runner?
*Thread Reply:* I think you have answered my question before but I cannot see the history from the free version of slack haha
*Thread Reply:* if you’re using wdio runner, check out their config docs for the version you’re using (If you’re <5, you’ll need to check the boneyard).
*Thread Reply:* in my company’s initial automation project, we had a config for that, but it’s so deprecated that it may no longer be the same 😉
*Thread Reply:* but ther is no otpion with wdio to re-run entire scenarios, there is only the option to re-run individual steps..
*Thread Reply:* I am using version <4 , v5 is not supported with wdio-cucumber yet haha
*Thread Reply:* hm, afaik, retry for the entire scenario is not readily available as a config option for cucumber. Ideally, you’d want to eliminate the flakes in the test, but obviously teams don’t always have the ability to do so if there are outside dependencies causing flakes 😉 I’m wondering if, in lieu of retry scenario support, one could capture the failing scenarios and kick off another cuke run using only those failed scenarios.
*Thread Reply:* Let me rephrase: One could do this. I’m just not sure what implementation surprises would be in store 😛
*Thread Reply:* I was able to have the test framework kick off a re-run of all of the failed feature files by adding a custom reporter in webdriverIO to track all of the failed features (feature file that contain failed scenarios), however I was not able to find a solution to have the automation re-run the failed scenarios only
*Thread Reply:* Do you mean wdio-cucumber-framework, or running cucumber with webdriverio under the hood?
*Thread Reply:* can you help me with the set up proces? i am running into alot of isseus setting up
*Thread Reply:* yes, so i am able to find my step definitiions in the project by ctrl- clicking them
*Thread Reply:* however when i try to run the project i get the error that my steps cannot be found, can you show you run configurations? This is the error i am getting telling me that my steps are undefined
*Thread Reply:* https://github.com/webdriverio/wdio-cucumber-framework#cucumberopts-options Read these and make sure you’re passing the step defs into cucumber properly
Hi everyone, have anyone used https://www.npmjs.com/package/scroll-js for the scrollnToView method and got it working? I need to scroll a long page to certain sections and assert text in one of my tests. I use selenium-cucumber.js
*Thread Reply:* @Cat : I don't know what that is. Can you give some context?
*Thread Reply:* cucumber-js docs have some info on this as well - you have to transpile your code if you’re wanting to use import etc.
*Thread Reply:* There’s a convo about something similar in this channel - just search in this channel for “json” - it was, I think, 1-2 weeks ago
*Thread Reply:* @Cat, I see, so instead of passing it from the Feature file what I can do instead is just import it direct from a variable.js file exported as json, which would work in my case since I am not looking for the clients to read it anyways.
*Thread Reply:* yep. And if you had multiple files you may need to switch between, you’d just pass in the name (or path + name) of the one you want
*Thread Reply:* What usually helps me when I bump into something that seems ridiculous to implement in cucumber is to step back and go “What am I actually trying to do as a user in this step?” That usually helps you spot the stuff that is too deep into implementation details rather than describing a business use-case step. if what you’re actually needing to do is denote that you are, say, interacting as a particular user profile (e.g., “I am logged in as an admin”), it makes a lot more sense than saying “I send POST /api/login with this JSON: {…}” Same concept 😉
*Thread Reply:* I actually follow the format of passing the name of the file in the feature file into the step
Given I post item "new jacket" into my cart
The step will then clean the string to be newjacket.json and look in the directory "/testData/item/newjacket.json"
It allows a step to be dynamic and allow user readability
*Thread Reply:* Thanks. I agree. 😛 https://cucumberbdd.slack.com/archives/C6QJ6N695/p1553098033054900
*Thread Reply:* If anybody wants to add this to the FAQ in the docs that would be really awesome! It seems to be a question we get more often (and then we can just point them there).
*Thread Reply:* @mlvandijk I’ll see if I can swing some doc updates soon-like
*Thread Reply:* Sidenote: (for strongly types languages,) I'd suggest using domain objects with builder pattern, rather than json files. I mean, why manage json files when you can have (compile time) checks on whether your objects are correct.
what do you guys think of this one? https://www.npmjs.com/package/cucumber-html-reporter
*Thread Reply:* I have used that one before. It's good but all depends on your requirements. Also the non javascript ones can still work if you have the jar I believe
*Thread Reply:* @Raymond, how did you generate the JSON report for this framework? after creating the JSON report, if you were to have a rerun of all of the tests, how were you able to merge the objects?
*Thread Reply:* I don't follow your question what do you mean merge the objects?
*Thread Reply:* @Raymond, let me clarify, so I run my tests one time and the cucumber-json reporter will create one json results object for me, and then say i have re-run logic implemented in my code so when I run it again, it creates a new JSON file for me. So far what I am doing is I am combining them using deep merge and using the result of that to create my html-report. However, the issue with doing that is the deep merge will create duplicates. I wondering if you have done anything similar in the past and if you have then maybe you can point me in the right direction
*Thread Reply:* @Raymond, I was able to find a solution for this issue by writing a custom report merger 🙂
*Thread Reply:* That wouldn’t be a pre-built config, precisely. We do something similar by abstracting out the drivers and swapping them in a before hook using tags. Less of a cucumber question and more of an abstract implementation question.
Hi!, I see that in (cucumber-jvm) JSONFormatter.java, the JSON can print "Keyword":"Scenario Outline" , when an Scenario Outline. But in (Cucumber-js) https://github.com/cucumber/cucumber-js/blob/master/src/formatter/json_formatter.js the Keyword for any scenario is hardcoded as "Scenario".
*Thread Reply:* Incorrect. Cucumber doesn’t care what version of wdio you use. The wdio runner that wraps cucumber, however, has not been updated for wdio v5 compatibility.
*Thread Reply:* I mean, not immediately because it’s a private repo, I’m on vacation, and I don’t have time to hop on and build a public example repo. ;) But in addition, if you’re using the wdio runner, my implementation is going to be entirely different.
*Thread Reply:* @Cat, no worries , I found a work around for this issue by simply just adding an external function even though it may be a bit of a dirty fix it seems to get the job done
*Thread Reply:* in the afterhook, how can i see if a scenario passed of failed? @Cat
*Thread Reply:* After({}, async function(scenario) { scenario.result.status });
*Thread Reply:* it would eliminate the need for the quotes in the feature file
*Thread Reply:* Ahh i didn't word my question correctly. I already have steps where i am using regex in the step def to read values from feature file.
The question above was, if there was a way to delimit a word in a string. but not show the delimiters in the runtime output.
*Thread Reply:* Hm. custom formatter perhaps? https://github.com/cucumber/cucumber-js/blob/master/docs/custom_formatters.md
*Thread Reply:* What's the end goal you're trying to accomplish? We know the what of what you're trying for, but that doesn't always equate to the end goal.
*Thread Reply:* If it's just for readability, are the quotes that bad? Could you spin it so that they're meant as an emphasis for what actually matters in the test?
*Thread Reply:* The end goal is to have the quotes (or any delimiter) out of it during the runtime. But i will need to check the value within the string matches the code table.
We have different screen, which use the same id for an button element, but depending on when the screen is accessed, the button name can change. The possible names for the button is stored in code table. same goes for any success, warning messages that apprear
*Thread Reply:* This is mostly for readability. For example, a message says "BID RATE should be greater than $1000". Here i want to check that BID RATE is in the code table for that screen. right now, i have "<<BID RATE>> should be greater than $1000" and i check the string between << >>.
*Thread Reply:* Right, I get that. But it seems like it would be a lot simpler to ditch the delimiters. It’s an awful lot of engineering to ditch them in the output when they could just be avoided entirely.
*Thread Reply:* @Cat That's true, i am not sure how else i would validate a partial value within a string..and the location is not the same on all messages
*Thread Reply:* That means i will have multiple steps depending on the location of the value in a string
*Thread Reply:* i was looking to create a common steps such as
And display success message as follows "BID RATE should be greater than $1000"
*Thread Reply:* which is what we have right now..except the value to be that needs check is between <<>>
And display success message as follows "<<BID RATE>> should be greater than $1000"
And display success message as follows "<<BID RATE>> is same as <<BID OFFER>>"
*Thread Reply:* yeah, you can feed into the stepdef using regex, then in the stepdef (or, probably more extensible, some helper(s)), you’d apply some further regex logic to the params in order to validate
*Thread Reply:* Getting more comfy with regex is definitely a great skill to have and I'm not even talking about looking ahead or anything like that. Cucumber School has a video on the basic things you'd need to work with Gherkin and it's a pretty short list. Expand on that a little with using something like regex101.com and you'll be pretty powerful!
*Thread Reply:* Also, is it truly valuable that the actual messages that are output are put in the feature file or are you more concerned that a process was successful? If it's the latter, then move the data check out of the feature file. Your step can then change to a more broad Then the update process will complete successfully and if you really want to, you can run an assertion against the message in a helper file called by the step def.
This way you're removing the headache you're currently facing, removing potentially unuseful data from whoever ends up reading the files, AND you're decoupling yourself from the specific message text at least somewhat. If you want to check the message, you can setup a json file or something and read that in or just do a string.contains(foo). Either way, it lessens the impact on future maintenance which is 👍 🙂
*Thread Reply:* If using a singleton pattern then why set it in the world instance? That will register it in every test which defeats the purpose of the pattern. I would recommend registering in the BeforeAll hook, which would mean every test has access to the Emitter
*Thread Reply:* I am using cucumber v4.2.0 and junit v4.12. Any suggestions please.
*Thread Reply:* Are you looking for a specific focus? There are some fairly straightforward examples like this one https://www.sitepoint.com/bdd-javascript-cucumber-gherkin/
*Thread Reply:* Because, it is an obvious option 🙂 It doesn't mean that it is not important.
*Thread Reply:* What - if anything - are you missing in the official documentation?
you could use a lib like this one: https://github.com/boblauer/MockDate
https://markus.oberlehner.net/blog/network-stubs-and-timers-with-nightwatch-and-cucumber/
Hey. I am running my tests with cucumber-html-reporter and I am unable to take a screenshot and attach it to be the report https://github.com/gkushang/cucumber-html-reporter Following the instructions, I receive the following error "world.attach is not a function"
*Thread Reply:* Make sure you are passing in params to your custom world instance constructor and then make sure to create this.attach = params.attach
*Thread Reply:* Yep that would be helpful. You may be able to override the value being used if it's being done on the frontend
@Nick Gonzalez https://cucumberbdd.slack.com/archives/C6QJ6N695/p1559840687040800?thread_ts=1559779719.027600&cid=C6QJ6N695
*Thread Reply:* @Nick Gonzalez no it would be in the CustomWorld constructor
*Thread Reply:* export class CustomWorld { protected constructor(params) { this.attach = params.attach; } protected attach; }
*Thread Reply:* My custom world is not a class. It is a function function CustomWorld(callback) { this.driver = new webdriver.Builder().forBrowser("chrome").build(); }
*Thread Reply:* Okay try the following function World({attach, parameters}) { this.attach = attach; this.parameters = parameters; this.driver = new webdriver.Builder().forBrowser("chrome").build(); } setWorldConstructor(CustomWorld)
Hey, I made a small sample project for how to use the pageObject pattern with cucumber.js. I thought maybe someone on here could use that as well. You can check it out on https://github.com/tobysoby/cucumber-js_pageObjects_example All feedback is welcome of course! 🙂
*Thread Reply:* @tobiasdoll You have put page objects under the feature directory, that is bad practice. They should go under a page directory or a step definitions directory
*Thread Reply:* @Raymond thanks for that. But why exactly is that bad practice? Can't all test code live in features? (Like in the official nodejs example, see https://github.com/cucumber/cucumber-js/blob/master/docs/nodejs_example.md)
*Thread Reply:* @tobiasdoll The feature folder should include only feature files and no ts/js/** code. The code such as steps / pages should live in their own folder as it muddys the water putting everything into the same folder.
*Thread Reply:* https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#formats Rerun
*Thread Reply:* is this something I can do with the wdio-cucumber-js framework? @Raymond
*Thread Reply:* @Leo can you link me that repository and I can take a look
*Thread Reply:* @Raymond https://github.com/webdriverio/wdio-cucumber-framework#cucumberopts-options
*Thread Reply:* I am going to try to pass the retry logic as a cucumberOption right to see if that will work for me, even though i couldn't find it in the documentation from that link
*Thread Reply:* this is what my cucumberOptions look like
cucumberOpts: { tagExpression: '@QA', format: [ 'rerun:@rerun.txt' ] } }
*Thread Reply:* so after doing alot of research yesterday, I figured out why it was not working for me. It is due to this issue: https://github.com/webdriverio/wdio-cucumber-framework/issues/6 , webdriverIO apparently blocks cucumberJS reporters, and because of that , I am not able to format the rerun.txt file. However, I was able to find a work around by having my automation retry entire failed feature files instead. It takes a bit longer, but definitely gets the job done. Anyway, thanks for your help!
*Thread Reply:* Short answer to "How do you people work with that?": I don't. 🤷 Long answer: Any time "my unit tests are in x so I want to keep using x with cucumber" is typed, god kills a kitten. When we write BDD tests like unit tests - to the point of wanting to retain our unit testing framework - it tends to result in tests that are also designed like unit tests, which lends itself to being nothin-but-anti-patterns when it comes to testing that is above unit level. My suggestion (and what I and my team do) would be to use an assertion framework that plays nicely with Cucumber (we use chai here). You can try to figure out something with jest, but imo, it's a lot faster to just use an assertion framework 🤷
*Thread Reply:* You're right, Cat. Faster and easier. Thank you so much for you advice.
Can the running order of cucumber-js hooks be configured somehow? I tried using what was suggested here, but without much luck... https://www.toolsqa.com/cucumber/execution-order-hooks/
*Thread Reply:* If I recall, before hooks run sequentially - after hooks are reverse sequence.
*Thread Reply:* So the top before hooks run first and the top after hooks run last?
*Thread Reply:* Yep. Have a look at the docs: https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/hooks.md
*Thread Reply:* There's no "official" offering, to my knowledge. There's a cucumber pre-processor for cypress.
*Thread Reply:* I see cucumber plugins for Cypress here > https://docs.cypress.io/plugins/index.html
*Thread Reply:* Here’s more info around official and other offerings of Cucumber. 🙂 https://cucumber.io/docs/installation/
*Thread Reply:* If I'm not mistaken, the gherkin can be in any language, but the step definitions would still call the same things (e.g., Given, When, Then). alternatively, you can use defineStep. But it looks like you're trying to import things that don't exist in the cucumber library.
*Thread Reply:* I thought, if my IDE is giving me that file, I can use it as it is.
*Thread Reply:* That looks like it might be a bug in either the IDE or the snippet generation. How was the file created?
JavaScript will allow this to work though! You can change line 3 to:
import { Given as Angenommen, Before, Then as Dann, When as Wenn } from 'cucumber';
😄
*Thread Reply:* Isn't is the other way around?
import { Angenommen as Given, Before, Dann as Then, Wenn as When } from 'cucumber';
*Thread Reply:* Ah seems on imports it's the other way around. I learned something today 😄 thanks @tooky
*Thread Reply:* See?! Everybody learns up in here! https://media.giphy.com/media/3og0IMJcSI8p6hYQXS/giphy.gif
*Thread Reply:* Screenshot at the last step or screenshot at the failed step?
*Thread Reply:* Screenshot at any point 😄 I am new to this and I do expirements
*Thread Reply:* I run my index.js file with node index.js and I get an error that takeScreenshot is not a function 😕
*Thread Reply:* Then you will need to parse the image data
imageStream.write(new Buffer(data, "base64"));
imageStream.end();
*Thread Reply:* here is a screenshot at failure implementation
if (scenario.status === "failed") {
await browser.takeScreenshot()
.then(async function (screenshot) {
decodedImage = screenshot
await saveScreenshotToLocal(scenario, screenshot)
})
}
export async function saveScreenshotToLocal(scenario, screenshot) {
const fs = require("fs");
const path = require("path");
const dir = "./e2e/reports/screenshots";
const now = new Date()
const hours = (now.getHours() < 10 ? "0" : "") + now.getHours()
const mins = (now.getMinutes() < 10 ? "0" : "") + now.getMinutes()
const sec = (now.getSeconds() < 10 ? "0" : "") + now.getSeconds()
const ts = now.toISOString().replace(/\T.**$/, "").replace(/-/g, "") + "_" + hours + mins + sec
//create folder screenshots, if it doesn't exist
if (!fs.existsSync(dir)) {
fs.mkdirSync(dir)
}
// Writing screenshot to a file
function writeScreenShot(data, filename) {
const imageStream = fs.createWriteStream("./e2e/reports/screenshots/" + filename);
imageStream.write(new Buffer(data, "base64"));
imageStream.end();
}
const featureFileName = ((scenario.scenario.uri).split(path.sep))
writeScreenShot(screenshot, featureFileName[featureFileName.length - 1] + "-line" + scenario.scenario.line + "-" + ts + ".png")
}
*Thread Reply:* So I have this taken from internet. Should I add ur part in there?
*Thread Reply:* So it must be a joke-app. I tried to download it, because it was like an enviroment especially for cucumber, and no matter how many times I click on Download it'll never does anything. So it may be a fraud 😛
*Thread Reply:* Should be covered in the docs - if you need something additional, look at npm for cucumber html reporter
*Thread Reply:* https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md
Ditch the global installation.
*Thread Reply:* So should I uninstall cucumber and re-download it but not globally
*Thread Reply:* global uninstall (if you aren't familiar with that, check the npm docs).
Install it at the project level - npm i cucumber or npm i cucumber --save-dev is fine
*Thread Reply:* then you can just follow the instructions from that doc I linked
*Thread Reply:* Sooo.. that worked! Now I want to make a html cucumber report and although I follow the steps at https://www.npmjs.com/package/cucumber-html-reporter I keep getting error Unable to parse cucumberjs output into json: './htmlreport/hellocucumber/nodemodules/package.json' Error: ./htmlreport/hellocucumber/nodemodules/package.json: ENOENT: no such file or directory, open './htmlreport/hellocucumber/node_modules/package.json'
Any suggestions?
*Thread Reply:* Unable to parse cucumberjs output into json: './htmlreport/hellocucumber/package.json' Error: ./htmlreport/hellocucumber/package.json: ENOENT: no such file or directory, open './htmlreport/hellocucumber/package.json'
*Thread Reply:* ... what? You don't need to add node_modules. npm creates that
*Thread Reply:* I'm learning 😛 But, still the main error keeps showing itself
*Thread Reply:* I would recommend deleting your node_modules folder, run npm i from the project directory, and then looking at the configuration docs for the formatter you're using
*Thread Reply:* FML I don't get it. I'll study more and come back to retry 😜
*Thread Reply:* nah, you just don't have muscle-memory for debugging npm yet 😂
*Thread Reply:* 😂 Anyway, thanks for your time and your help, I really appreciate it! Farewell Cat 😻
*Thread Reply:* Just check npm for the reporters. The README will be pretty straightforward
*Thread Reply:* you could just run npm run yourTestScript && open yourHtmlOutput.html
*Thread Reply:* (you'd need to use npm-run-all dependency if you want to execute that from your package.json)
*Thread Reply:* Well, technically I want to implement in js this scenario: Scenario: Given that I have gone to the intelligencia page When I click on Request a Demo Then I should see a picture
And I can't do the Given part, when I use click() , i am getting error click is not a function
*Thread Reply:* Cucumber doesn't give a shit WHAT you're doing 😛 you have to tell it what you're doing.
*Thread Reply:* Yes, but not in cucumber because cucumber does not click anything. 😛 I can tell you how to click in webdriverio
*Thread Reply:* // #/Users/dimitrisliaropoulos/Desktop/htmlreport/hellocucumber/features/step_definitions/stepdefs.js var {setDefaultTimeout} = require('cucumber'); setDefaultTimeout(60 ** 1000); const assert = require('assert'); const { Given, When, Then } = require('cucumber'); const {Builder, By} = require('selenium-webdriver'); let driver = new Builder().forBrowser('chrome').build(); var chrome = require('selenium-webdriver/chrome');
function clicky(){ document.getElementById("nav-request-demo").click(); }
Given('that I have gone to the intelligencia page', async function() { await driver.get('https://www.intelligencia.ai'); }); // When('I click on Request a Demo', async function() { await driver.getElement(By.linkText("nav-request-demo")); }); // by class? Then('I should see a picture', async function() { driver.findElement(webdriver.By.class("hero-img-2 section")); });
*Thread Reply:*
function clicky(){
document.getElementById("nav-request-demo").click();
}
you're using document and not driver which probably doesn't help 🤷
*Thread Reply:* I was using diver berfore 😛 then i googled searched. I saw the document.blahblah and threw it in
*Thread Reply:* I'd suggest reading the API docs: https://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/index_exports_WebElement.html
*Thread Reply:* Hi @Alexander Foxleigh - the reason you cannot use flags is that the parameter type’s regexp will be merged into a bigger regexp covering the entire step text. Allowing flags such as i would then apply to the entire regexp, which would be undesireable. For example there might be another parameter type for currencies with regexp [A-Z]{3} and we don’t want that to match gbp, only GBP.
*Thread Reply:* As for the error you’re seeing - I’m not sure. What Node.js version are you running this on?
*Thread Reply:* Also, what version of cucumber-expressions do you have in your package-lock.json or yarn.lock?
*Thread Reply:* In order to help you and fix this bug it would also be useful to see a stack trace. Please create a new issue at https://github.com/cucumber/cucumber/issues
*Thread Reply:* The more detail you add, the higher the chances of someone picking up the issue to fix it.
*Thread Reply:* If you submit a pull request with a test that reproduces the bug that would be great!
*Thread Reply:* @Aslak Hellesøy I think the problem is that the team I'm on are using the cucumber-boilerplate library which inexplicably has 3 different versions of cucumber-expressions installed, plus there is a lot of black-box stuff going on behind the scenes which I can't debug. So until I can bully them into dropping boilerplate, I think we're stuck, I'll just have to work around it.
*Thread Reply:* Yuck! I see they reimplemented the training wheels I tossed 8 years ago. This stuff just won’t die! https://aslakhellesoy.com/post/11055981222/the-training-wheels-came-off
*Thread Reply:* No, the step definition is defined once in the code. It will execute when it is called, like any other function.
*Thread Reply:* Error is pretty self-explanatory. The function is timing out (aka the step is taking too long to complete) either extend the timeout of the individual step or of the overall timeout
*Thread Reply:* ok, so, those aren't strictly JS questions - they are questions specific to the driver you're using. You would need to look for similar groups for that specific type of driver (e.g., wdio, selenium, puppeteer, etc.)
*Thread Reply:* If your web driver fails to click an element, you should get an error. Also, instead of taking a screenshot alone, you should actually assert something. 🤷♀️:skintone2:
*Thread Reply:* The state of the application? You could assert based on the visibility or existence of an element, etc.
*Thread Reply:* It's fine to take a screencap, but that doesn't actually tell your tests if they should pass or fail (although they will if it fails to screencap)
*Thread Reply:* for our tooling, we'd do something along the lines of expect(this.driver.isVisible(myElement).to.be.true;
*Thread Reply:* for that use case, wdio would get the visibility state of the element being passed in and, if it returned true (visible) the test would pass. If it returned false, the test would fail.
*Thread Reply:* @Loki every release has a changelog where you’ll find the answer: https://github.com/cucumber/cucumber-js/blob/master/CHANGELOG.md
*Thread Reply:* with version 0.9.5 we were able to invoke parallel execution but report generation on execution impacted as it started overwriting with the last instance execution
*Thread Reply:* To my knowledge, no. That's a pretty huge version gap - I recommend looking through the changelog at breaking changes to see what would need to be altered to get from 0.9.5 to current.
Adding some more context 🙂
*Thread Reply:* Update: ^ haha thanks, also so maybe I am just not understanding how allure works in general, but when I use the allure command line "allure generate" the report does not generate but when i do "allure serve" it generates the report correctly.
Other than allure for cucumberJS integrated with webdriverIO v5, is there any other reporters currently available?
*Thread Reply:* Cucumber doesn't care what application framework you're using. It just runs the tests. Whatever you tell it to do in the step definitions is what it will do.
*Thread Reply:* In case of Dotnet i have used Spec flow and in the step definition file i have used Selenium. If application is of Sencha what can i use in Step definition.
*Thread Reply:* You would still use selenium unless you needed to use something else. Whatever you put in the step definitions is entirely dependent upon what and how you're testing - not so much of a cucumber question.
*Thread Reply:* cucumber-js ../../tests/features --require=./tests/step_definitions/**.ts --require=../../../../../tests/lang/ts/step_definitions/**.ts --format progress-bar --require-module=ts-node/register
*Thread Reply:* per docs and my own code, that flag is --require <myPath>
*Thread Reply:* not sure if it would affect it, but why aren't you in the TLD when running?
*Thread Reply:* in that case, I'd confirm the path. Nothing else should be impacting, from what I can tell
*Thread Reply:* but if I move the top level step definitions to my local package it works
*Thread Reply:* @luke I can't actually because I'm using specific dependencies from the package
*Thread Reply:* root
- packages
-package1
-tests
-src
-package.json
- package2
- tests -step_definitions
- package.json
When I run my tests from package1 I cannot use step_definitions from the top level tests
*Thread Reply:* yes you can, just require something at a higher level than you're at.
*Thread Reply:* ok but I keep getting this error TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received type undefined
I tried with something very simple:
import { Given } from 'cucumber';
Given('I have a workspace', async function(): Promise<void> {
...
})
*Thread Reply:* in the stacktrace I can see that the error point on the Given(...) line, but I think it has nothing to do with my code because it's working correctly if I move that file in my package
*Thread Reply:* I have the same error if I run from the TLD, btw when I use --require-module=ts-node/register does it use the local tsconfig.json file?
*Thread Reply:* But meant more to illustrate that using mocha within cucumber is a bit superfluous
Hey, guys! Can we please review this PR with this cool feature for cucumber-js community? https://github.com/cucumber/cucumber-js/pull/1205
*Thread Reply:* I surprisingly can't see access to an id but you can build one based on the scenario such as
*Thread Reply:* This will output something like
spec/cucumber/features/quoting/quotes/modeling/DisplayQuoteCostsData.feature119 which you could then massage into a format you would like for an image name
*Thread Reply:* Also I tested with scenario outlines and the line is unique to each example
Hello, anyone who can help me with my question? Thanks in advance. 👇
*Thread Reply:* I was crossing fingers not to get a response like this 😞
*Thread Reply:* So in this screenshot, you can see that the section I marked in the yellow rectangle corresponds to node_modules folder.
The rest of it corresponds to a bot I defined. But the flow that goes through the page objects before getting to the bot is not included in the stacktrace. That’s what I’m missing and that’s what I need
*Thread Reply:* I guess I'm a little confused as to what kind of stack trace you want? This stack trace looks correct to me. From bottom-up has the steps to the root of the failure (selenium failure on a wait)
*Thread Reply:* So let’s say I have a login test case. The architecture I have is:
- Gherkin
- Step definition
- Login Page Obejct
- Bot Style (to interact with the browser)
*Thread Reply:* So the stacktrace I’m expecting is for instance, something that includes what’s in the screenshot above, but then the stack continues telling that from step definition on line 3, it went to page object on line 36 and then it went to bot on line 17 and that’s where it crashed
*Thread Reply:* Depends on how your code is referencing the other pieces of the architecture.
The prototype is here: https://github.com/cucumber/cucumber-engine
I’d be happy to help you if you have any questions. And we’ll pay you if you submit expenses to https://opencollective.com/cucumber
*Thread Reply:* @Jonas Amundsen for step 2) I recommend you grab the code in gherkin/javascript/lib at tag gherkin-v5.1.0
*Thread Reply:* Don’t forget to ping if you get stuck too! We’re here to help if we can 🙂
*Thread Reply:* Matt, I've posted a PR with some preliminary work if you would like to take a look: https://github.com/cucumber/cucumber/pull/689
*Thread Reply:* Hey @Jonas Amundsen that looks impressive! I will try to review this in the next couple of days. Thanks so much!
*Thread Reply:* https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/world.md
*Thread Reply:* and more details on attachments: https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/attachments.md
*Thread Reply:* You're using an arrow function. It impacts this which is necessary for hooks, steps, and the world instance to operate properly. Change it to async function () {...} and it should work
*Thread Reply:* should get it working, but if it doesn't, lmk and we can dig more 🙂
*Thread Reply:* @Cat interesting. It did work. Why is it that when using arrow functions it doesn’t work as expected?
*Thread Reply:* Hi, it's cukebot again. "Doesn't work" is rather vague. What are you seeing and what had you expected to see?
*Thread Reply:* It has to do with the way arrow functions bind lexical this, which impacts this which is shared between the world, steps, and hooks (as mentioned above).
*Thread Reply:* I know of AfterAll(), but I’m using protractor which uses webdriver js. And because of the way webdriver js handles things, AfterAll() is triggered after every single .feature file, rather than running after all files have run
*Thread Reply:* Then I would check with protractor folks 🤷 It's going to be distinct from using cucumber itself. It's not actually a webdriver js issue, iirc, but rather that it's a cucumber plugin for protractor, rather than cucumber running protractor 🙂
*Thread Reply:* You can use YAML or JSON w/JS - cucumber is for demonstrating behaviors, however. You wouldn't reference the fields the way you're doing here. What precisely are you attempting to do with the data here?
*Thread Reply:* The idea is that product owners can see what is being involved in the tests, as they do not know the code, it would cost them a little more to understand.
*Thread Reply:* Why would you not simply use business language for the data tables?
e.g., I insert <name> in the name field where the data value for name is, say Steve (or whatever value, really)
This would then pass into the stepDef function, which you then would tie to datosPrueba.comment
*Thread Reply:* Hi @Ryan Kazokas we use Angulars own In memory api tool, its very extensible and has covered all our requirements so far.
I've hooked in an environment variable so that we can flick from using the mocked api to the real ones while developing.
John Papa uses it in the tour of heros demo, I've taken that and build cucumber into it in this repo as a proof of concept: https://github.com/DesHorsley/angular-tour-of-heroes
*Thread Reply:* Awesome, thanks for the advice! I will definitely take a look at this!
*Thread Reply:* @Des Horsley Almost thought we'd be able to use this, but we can't since we have an SDK that is using axios and is not using httpclient to make the requests. From what i can tell it will only work with the http and httpclient
*Thread Reply:* I was unable to find anything that would aid in this outside of running an actual mock server, b ut this has it's limitations, mainly running tests in parallel is the main concern with that
*Thread Reply:* Agreed. I know java has good support as I use cucumber there, but i think the main problem is that unlike junit, protractor isn't really made for mocking/stubbing but more for browser automation.
https://developers.google.com/web/updates/2017/04/headless-chrome
*Thread Reply:* Was able to get this working again this morning by following https://intoli.com/blog/running-selenium-with-headless-chrome/ which was referenced in the link @Gem provided. I inherited this project, so I deleted all the previous chrome setup and replaced with this much simpler version
*Thread Reply:* @mlvandijk thanks I've seen those but I was looking more for a tutorial to code along with to learn.
*Thread Reply:* What do you want to learn? There's also Cucumber School if you want a course on BDD / Cucumber.
*Thread Reply:* I was trying to find tutorials with BDD and JS, that site only has Java or Ruby
Hey, folks Just noticed that the new version of cucumber doesn't work with cucumber-pretty module. need to update the cucumber-pretty module based on the latest event protocol methods. https://github.com/kozhevnikov/cucumber-pretty/issues/10
*Thread Reply:* @Jonas Amundsen yes you should be able to. If not it’s a bug 🙂
@Jonas Amundsen it uses gherkin 5 (https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/blob/bfdb5af466a2853937ecd2b209eb6129a79caeb9/package.json#L54), which might have a bug with this. I can’t find any mention of fixing this bug in the changelog (https://github.com/cucumber/cucumber/blob/master/gherkin/CHANGELOG.md) - if it was ever reported. Maybe the bug is still there, IDK.
Does gherkin handle replacement though? The tool (cypress-cucumber-preprocessor) performs replacement by itself, as seen here [1] and here [2]
[1] https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/blob/v1.16.2/lib/createTestFromScenario.js#L10-L14 [2] https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/blob/v1.16.2/lib/createTestFromScenario.js#L109-L117
PR: https://github.com/cucumber/cucumber/pull/292 - released in 5.1.0, which is what cypress-cucumber-preprocessor is using.
@luke they are on gherkin 5.1.0 - https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/blob/bfdb5af466a2853937ecd2b209eb6129a79caeb9/package.json#L54
*Thread Reply:* I'm not seeing --require anywhere. How are you telling cucumber where the step definitions are?
*Thread Reply:* because it looks like it's just passing the formatter params
*Thread Reply:* I am using tags to identify the features in which to run, but even still, when I run npm test, it should run the entire suite
*Thread Reply:* ok. let me rephrase that.
in your package.json, how is npm test defined?
*Thread Reply:* Right. that is just telling cucumber the reporter you want to use. It's not telling it where the support code is.
*Thread Reply:* https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md
*Thread Reply:* yeh most of them you need to provide the location of step defs. I know some of the flavours have autoloaders if you put the steps in a certain place.
*Thread Reply:* i'll use whatever really. I just want something where i can do basic browser automation.
*Thread Reply:* we'll move on from that once i lose the training wheels. The problem I have is I only really understand ruby.
*Thread Reply:* I could throw a quick GH example up for you but realistically prob won't get time until later this evening (EST)
*Thread Reply:* that's fine. I tried wdio, I couldn't get it to work. I tried wdio (both selenium and non-selenium), i tried raw selenium (worked best but had some issues), cypress (not great), and all of them I couldn't reliably get to run.
I'm fairly certain I'm the issue here.
*Thread Reply:* Nah, config + getting started is, hands down, the hardest shit 😄
*Thread Reply:* Kind of. Literally all i'm looking for is a sample to do page object model testing with js. The problem I have is it feels like the config files in js are 100s of lines compared to ruby which has 5-10 lines.
*Thread Reply:* https://github.com/site-prism/site_prism/blob/master/features/support/env.rb
That's the config file for site_prism (And some of that isn't needed for regular testing, its because its using travis)
*Thread Reply:* as someone who went from Java/Groovy/Lisp code base to Javascript/Golang codebase: Forget everything you know about any other language before you proceed 😂
*Thread Reply:* if I can get a directory that has 1 feature, that loads up google and does a search and validates the results that would be epic. I know it sounds silly but I'm genuinely struggling to do that. But I know it's me that's the issue.
*Thread Reply:* Think lisp is similar to ruby in some aspects, or at least thats what matz (the creator), said.
*Thread Reply:* no no, it's not you - even for experienced JS devs, getting configured and moving with browser automation is difficult
*Thread Reply:* im leaving work soon and won't be back until wednesday and plan on having a break. So don't sweat it.
If you can give me some pointers epic. But don't stress yourself too much.
*Thread Reply:* no biggie! let me see if I can whip up just a basic config + cucumber + wdio example for you so you can see the gist of it
*Thread Reply:* take your time, but yes please feel free to help.
I try and do the same with ruby, but sadly it's getting less popular 😞
*Thread Reply:* @luke did you ever get anything to work? im in the similar boat with JS and trying to learn
*Thread Reply:* I'm at work right now, but can whip up something pretty quickly once I get some free time (which I hear is a real thing, in spite of it being slightly mythological in my own personal life)
*Thread Reply:* i haven't been able to get anything to work except cypress, so...i feel dumb about it all
*Thread Reply:* I tried nightwatch, but it kinda isn't right. Because it wasn't quite doing proper browser automation.
*Thread Reply:* well it is fine, it's just I found the configurability a bit too coupled. Frankly I don't care how I write the tests, so long as I can write some and maybe learn by doing it.
*Thread Reply:* @Amanda R you said you couldn't get anything to work in cypress - could you elaborate?
*Thread Reply:* yeah @Cat Cypress is the only thing i can get up and running
*Thread Reply:* ive tried webdriverio and also nightwatch without any luck
*Thread Reply:* ive worked with selenium at a past company but never had to deal with project set-ups
*Thread Reply:* my recommendation:
Start with cucumber. get it set up doing a basic scenario sans any other configs (e.g., get some steps like Given 2 / When I add 2 / Then I get 4)
*Thread Reply:* Once you get that done, start with a very basic config to start your driver
Yes indeed. And here is a proposal for how we can better keep track of features that should be ported around: https://github.com/cucumber/repo-template/issues/4
I've tried to outline the above-mentioned proposal here btw: https://github.com/cucumber/cucumber/issues/768
It's me again, the cypress guy. So, the way cypress-cucumber-preprocessor works today is that it transforms the feature-file by parsing it using the old, native parser, and returns a template containing the AST [1], which uses Cypress' own test-constructors [2] (describe & it - from Mocha) to create tests. Since it works on the AST, it needs to re-implement EG. the expansion logic of scenario outlines. Consuming pickles would remove the need for re-implementing that specifically, but the pickles themselves doesn't contain the necessary information to re-construct the hierarchy of describe() and it() blocks. Any of you have any suggestions as how to best integrate these tools?
[1] https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/blob/v1.16.2/lib/loader.js#L16-L19 [2] https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/blob/v1.16.2/lib/createTestFromScenario.js#L37-L51
you can use chromedriver package from npm https://www.npmjs.com/package/chromedriver
I’ve created an issue here @kerbe: https://github.com/cucumber/cucumber/issues/807
@charlierudolph, are you already working on this btw? Ref. https://github.com/cucumber/cucumber/pull/814
I created simple repo with freshly initialized react native project with expo-cli @Aslak Hellesøy, it can be found here: https://github.com/kerbe/rn-cucumber
It is in working state, meaning if you clone and npm install, you should be able to run npm test and get a start. However, this is point where right way is lost for me... That App.jshas that View and Text elements, and I would like to test that those exists. But if I try to require/import App.js to features/stepdefinitions/mainscreen.feature, I get errors, and I'm lost.
*Thread Reply:* We have been trying to distill what we learned into Feynman: https://github.com/cucumber-ltd/feynman
*Thread Reply:* I don’t know if it has enough docs yet, or how usable it is for people who didn’t work on it yet, but feel free to have a poke around.
*Thread Reply:* @Jon Beller I’m writing a blog post series about it.
*Thread Reply:* What’s stopping you from using Serenity, out of interest?
*Thread Reply:* I may have this wrong, but it looks like Serenity would be difficult to incorporate into my existing test suite that tests an electron app using Spectron, since it has Protractor baked in.
*Thread Reply:* Honestly, you can build your own screenplay framework in a few lines of code.
*Thread Reply:* See here for an example: https://github.com/mattwynne/screenplay-example/blob/b31cf1b3731b63a6d0da5c8912888f7823052749/features/step_definitions/steps.js
*Thread Reply:* Again, please ask me anything: I’m working on these series of article for someone just like you, so it would be really helpful to me to understand more about your context, what questions you have etc.
*Thread Reply:* @matt-SmartBear this is really exciting, I've been using blue harvest with Angular and its worked out well, but it looks like the project is no longer supported. Its been working well with cucumber, but still needs love. I'm glad to see that work is being done on an alternative.
*Thread Reply:* https://github.com/angular-seattle/blue-harvest
*Thread Reply:* In this example, what is on this.abilities? It looks like name and app. Why do we treat those as "abilities"?
*Thread Reply:* OK @Jon Beller so in the screenplay metaphor, the abilities are “things the actor is able to do”. In this example, the actor has the ability to:
• recall their own name
• make calls to the app’s JavaScript API
Other abilities you might want to pass to the actor are things like a browser or a databaseConnection
*Thread Reply:* I have a concern that the Tasks aspects of the pattern would make our tests harder to debug compared to putting procedural logic directly in step definitions.
With procedural logic you can step through line by line with the debugger. In screenplay, your interactions execute in a loop inside attemptsTo, and then are hidden under layers of Tasks you'd have to step into and back out of.
Have you found this to be an issue?
*Thread Reply:* I'm also concerned it will be tempting to shove one-off scenario-specific logic or conditionals into Tasks, making it difficult to reuse them, or to know what you're breaking if you modify them.
*Thread Reply:* re. debugging, to be honest I haven’t notice that to be much of an issue, no. Would it not be possible, depending on where you put your attemptsTo function (you could use https://github.com/cucumber-ltd/feynman/) to have your debugger step over the infrastructure and stay in your task functions?
*Thread Reply:* I think it’s generally OK to have quite scenario-specific one-off tasks. What I see happen is that those tasks tend to get decomposed into smaller re-usable pieces, either tasks or actions. The smaller those pieces are, the less likely it is that you’ll want to modify them in future, so there’s less risk of that “what will I break if I change this” problem, I think.
*Thread Reply:* I might be good to talk about a specific example here. Do you have one?
*Thread Reply:* If you look this channel upwards, in beginning of December, might be easiest reminder?
*Thread Reply:* https://cucumberbdd.slack.com/archives/C6QJ6N695/p1575470121133000 << like that was last time it was talked about
*Thread Reply:* And original problem was this: https://cucumberbdd.slack.com/archives/C6QJ6N695/p1574794814100000
*Thread Reply:* It relates also to this issue: https://github.com/cucumber/cucumber/issues/807 as hopefully result of my problem is solving that issue, ie. providing good example how cucumber-js is used with react native.
I believe https://github.com/istanbuljs/nyc would work
*Thread Reply:* thanks, but I was hoping for something else. I already tried it with jest which include nyc, but I will give it another try with mocha
*Thread Reply:* Hmm why did it not work? I don't think there should be a difference between jest and mocha in this case I haven't run it with jest but I did run it with Mocha and it worked, heres an example of the script I ran
"test:coverage:cucumber": "nyc --nycrc-path=spec/unit/.nycrc-v1.json npm run cucumber:regression"
• test:coverage:cucumber - What I named my npm script
• nyc --nycrc-path=spec/unit/.nycrc-v1.json - Calling Istanbul and the profile
• npm run cucumber:regression - My cucumber tests
*Thread Reply:* sorry I should clarified what my issue really is. I am trying to integrate cucumber js for my bdd testing. I was able to do that with a simple create-react-app typescript template. I am trying to achieve two things here. 1) create a json report for my bdd feature files (was able to do that) 2) create a code coverage report(inprogress of figuring it out). I know jest can give me a code coverage report, but it doesn’t integrate well with cucumber-js
*Thread Reply:* I guess I don't understand, what you are saying. If you have cucumber working and test running then you just need to create an istanbul profile of what code you want it to look at and what level of reporting. Then you just call your test run with nyc
nyc *nyc-profile* *process to run*
In this case the process to run is your test suite ^ that could be jest, mocha, cucumber etc
*Thread Reply:* thanks I think I figure out what you are saying. Sorry I am still new to testing. Thank you for guiding me through this.
*Thread Reply:* No problem, feel free to ping here again if you have more questions
*Thread Reply:* @Vu could you share how you have setup your cucumber-js + react native? I tried doing similar with expo, but didn't get easily cucumber-js working. Your setup steps could give me necessary direction what to do, so I'll get forward too. 🙂
*Thread Reply:* @kerbe I didn’t used react native, but react. All I did was install cucumber and set the script in my package.json.
*Thread Reply:* Ah, my mistake. I somehow managed to read / understand that you had used react native. 🙂
*Thread Reply:* yeah sorry I never used react-native, I don’t know how big of a difference between the react and react-native configuration
*Thread Reply:* Too big for cucumber-js to "just work" like it does with react. 🙂 As I haven't used that much of react, seeing talk about Jest equalled in my head that it is react native project. 😄
*Thread Reply:* create-react-app script used jest as their default test-runner. I just simply replace with cucumber-js
*Thread Reply:* Did the coverage work? Also you shouldn't need ./node_modules/.bin in your scripts as npm knows to look in there for commands, and they default to using the local libraries
Has @matt-SmartBear or @Aslak Hellesøy had chance to check my problem yet? Reminder, it's getting cucumber-js to work with React Native, specifically with Expo. Basic repo for problem: https://github.com/kerbe/rn-cucumber
*Thread Reply:* Highly recommend the cucumber-js specific docs: https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#parallel-experimental
*Thread Reply:* (ignore the first link, we were debating how to dress for a work event)
*Thread Reply:* Thanks @Cat I had come across that. I'm not sure how to translate that into my protractor setup, my undstanding is that angular kicks it all off under the hood.
I've updated my cucumberOpts in protractor.conf.js to include the following (Just to cover my bases):
PARALLEL: true,
TOTAL_SLAVES: '10',
parallel: true,
total_slaves: '10',
format: [
'json:reports/e2e-reports/results.json'
]
If I don't pass in the total slaves as a string I get an error about an unexpected path.
Running the tests with the above code doesn't seem to provide any time advantage, or produce multiple outputs which I thought would happen (One for each process?)
Any thoughts?
*Thread Reply:* ahhhh, if you're kicking it off with the protractor runner you will want to specifically look at the protractor docs. It may or may not be supported.
*Thread Reply:* Thank you for responding. I'm using Protractor and Cucumber together and it seems really difficult to parallelize tests with that combination. My idea is to gather the list of scenarios which will be run, split them up evenly, create several cucumber "rerun" files, and then run my tests using the cucumberOpts.rerun option. This way I can parallelize them evenly instead of just using shardTestFiles which breaks the tests up by feature file.
@Yehuda Miller haven’t tested but you might be able to:
• use the —dry-run flag to stop the steps running for real • write a custom formatter to build your array of scenarios https://github.com/cucumber/cucumber-js/blob/master/docs/custom_formatters.md
Would be interested to hear the motivation for this
*Thread Reply:* This isn't a cucumber question. This depends entirely on what you're using as a driver.
*Thread Reply:* Our steps are written in gherkin. Does that help? Sorry, I'm very new to this system.
*Thread Reply:* This sounds like a question about Selenium WebDriver @Adam - they have their own community
*Thread Reply:* Thanks. There's so little documentation with this system we're using that I'm having to run in circles to get any information.
*Thread Reply:* If it's in js, you can look at the package.json to see what dependencies are being leveraged
*Thread Reply:* @Adam Cucumber runs the tests for you (and reports the results), but something will be being called from the javascript code in those tests that is automating the browser. Could be selenium but could be something else too. What kind of app is it you’re testing?
*Thread Reply:* I totally get that and I didn't mean to imply anything. "No" is a perfectably acceptable answer here.
*Thread Reply:* to use cypress with cucumber, you have to use a pre-processor. We opted to simply not use cypress with our cucumber implementation, honestly. It wasn't worth the hassle.
*Thread Reply:* I've got some sample project here: https://medium.com/@itortv/how-to-integrate-cypress-and-cucumber-in-your-development-flow-in-just-a-few-weeks-96a46ac9165a which seems simple enough.
I'm still struggling to get something that combines the 2 aspects I previously had which both worked.
A seleniumJS raw script which I could pause and debug. And proper tests written with POM and classes / functions e.t.c.
*Thread Reply:* I have this project that does POM: https://github.com/healthbridgeltd/ui-tests-boilerplate
*Thread Reply:* I'll take a look at that @Ayethin - I think one thing I really need is just a mini-sitdown session to get it working the way I can get stuff working in ruby, So debuggable / viewable e.t.c.
*Thread Reply:* yeah, that is slightly out of date, but should hopefully help
*Thread Reply:* I'm struggling to get cypress to pause during execution in my code in RubyMine - Raw Selenium works, so it's either something simple I'm doing wrong or some misunderstanding I have.
*Thread Reply:* you can pause with .debug() or just from the browser itself with the built in debugger if that is what you are after. Outside of that I don't know any way
*Thread Reply:* neither are working properly. Because I'm still a newbie. I think I've got this project 75% working. There was some bugs in the selectors.
*Thread Reply:* Big issues I've got atm.
Output response Speed Config debugging
*Thread Reply:* to my knowledge, debug() is the way to go for that.
That said? Even with an army of expert frontend developers well-versed in automation infrastructure, we've had TONS of issues with even traditional mocha cypress implementation stability so, frankly, the issue very well may not be on you =|
*Thread Reply:* I just mean using my IDE and putting a red stop sign on a line of code. then clicking play,.
*Thread Reply:* it works in ruby (obviously), but it also works in raw selenium code. But for some reason using cypress or other stuff makes it not work. I've not got the knowledge to work out why.
*Thread Reply:* Also I'm not comfortable in changing cypress config yet (let alone on the fly), so I'm testing in electron which is super bad.
*Thread Reply:* I managed to get the cypress cucumber one working quite reliably now. The main issue I have now is that of debugging / traceability.
I.e. when I run it outputs an xml file with no stacktrace (or a complex one), I want a nice easy stacktrace to work out what line is failing, and also a way to get things like logging to file and browser config working. But I made ok progress.
*Thread Reply:* I'm guessing this is no longer available? I can see the code in cucumber.js but it doesn't appear to be accessible
*Thread Reply:* Still struggling with this (https://github.com/cucumber/cucumber-js/issues/1489) @Aslak Hellesøy
*Thread Reply:* Real talk:
I don't 🙈
We have a @wip tag for running scenarios/features under active development and I just hit that script. That said, it's mostly because we just haven't had time/prioritization to fiddle with a watcher. One of the FEs in my org would probably have a much cleverer solution 😁
*Thread Reply:* Honestly I never even thought of implementing a watcher, but what I would recommend using if you wanted to add a watcher is using gulp https://gulpjs.com/docs/en/api/watch This will work with typescript too
*Thread Reply:* Generally speaking, passing data from one scenario to another is considered an anti-pattern. Ideally, you want to keep them as separated as possible. Could you give an example of the sort of use-case you're trying to implement?
*Thread Reply:* @jradom https://stackoverflow.com/a/61241558/1267688 here is an example of how to pass data between steps. As for passing between scenarios, it's possible but not recommended because it violates test isolation between scenarios.
*Thread Reply:* First of all: no need to post your question twice 🙂
You need to install it through apt install tree. You might need root permissions (prefix with sudo, sudo apt install tree.
I cannot answer your question related to Windows.
*Thread Reply:* It was a copy paste from the other channel. the backticks didn't make it 😊 updated now
*Thread Reply:* I had to fiddle with this a bit when I trying to run cucumber directly, but now I have picklejs in the mix and that seems to run cucumber from cypress, so I might not even need that at all anymore. I'll take it out now to see if that even matters.
*Thread Reply:* More gracefully than...? You have a few options built in (for example: setting the timeout at config level), and there are a few things you could implement yourself.
*Thread Reply:* Usually it's helpful to identify long-running scenarios and either break them into shorter scenarios or identify the underlying cause of poor performance and remedy it
*Thread Reply:* Hi Cat, some more details: Our tests are a little unique, in that we’re testing videos, which can take some time before we make our assertions. So, we necessarily have some cucumber steps where we start something happening and we simple have to wait some time.
*Thread Reply:* One way I found to stop a test is in the world, I listen for a stop event. If one occurs, I throw a “Test stopped” error in the world itself.
*Thread Reply:* Hm. So basically
Given I go to MyPage
When I click MyVideo
And I wait for the video to play
Then I see <whatever the completed state is>
Right?
So when you say you're listening for the stop event - do you mean the video has unexpectedly stopped before it has completed? (trying to make sure I understand the scenario)
*Thread Reply:* If that's wrong, you can send me a (sanitized, if you need to) scenario so I grok it and I can help 🙂
*Thread Reply:* Hi Cat, your scenario is pretty close, so let’s go with that. So, let’s say that scenario is running, and currently we’re waiting for 10 minutes worth of video to play.
We have some UI that let’s us monitor running tests. In some cases, we might have to cancel the test manually. So, we have a button on the UI, and we can send a stop request to our server.
*Thread Reply:* So, we’d like to be able to stop the test from outside cucumber, but still allow it to execut after hooks
*Thread Reply:* Hm. To my knowledge, canceling the entire process manually would not allow you to execute the after hooks since it would end the entire process. However, you could ditch the after hooks and implement them as before hook cleanup and bypass the entire issue potentially
*Thread Reply:* alternatively, you could write an entirely separate script that runs independently that could be executed manually upon manually stopping the tests.
*Thread Reply:* okay, will look into that, thanks. sounds like there’s no way natively in the library to “interrupt” the steps. thanks for your time 🙂
*Thread Reply:* Short of setting a timeout or conditional stop, no, not that I can think of off the top of my head - that would not work with the manual stop use case though =|
*Thread Reply:* I've written automation with video and it can suck given the set time restraint based on video playback. 😞
*Thread Reply:* If possible, I'd recommend trying to use short videos wherever possible. Also, if you know the length of the video, you can put in a wait for the duration before moving on so that if things hang, you're not sitting indefinitely at least.
I know always using short sources isn't possible, especially since some issues only present themselves in long running sessions.
*Thread Reply:* Thanks, yeah that’s exactly the issue. We do use short videos when possibly. But we’re actually measuring latency drift in long running videos, so that’s where this scenario comes up. Sometimes we actually have to let the video run up to 3 hours!
*Thread Reply:* @ec-mark this might sound crazy… and I’m a complete amateur to your domain… but….
I wonder if an automated test like this is the best way to measure something like that.
Is it possible to think of a way to test latency drift in production?
I mean instrument the app in someway to provide you the data you need while users are actually playing videos? You could then have alerts if things start to go out of the boundaries you set for normality.
*Thread Reply:* ^^ That would actually be my suggestion as well, @tooky. It's how I would measure poor performance in, say, a long-running date aggregation. I don't see why the same principle couldn't be applied to videos.
*Thread Reply:* I am not using any other framework…using cucumber for writing automation tests and have own library built for invoking rest apis
*Thread Reply:* Ok, i figured this out…we can achieve this using profiles in CLI…Thanks you !!
*Thread Reply:* Not directly in the .feature file, but you can accomplish that in the steps files, which are js files
*Thread Reply:* Thanks Marco.. can you share any references to follow?
*Thread Reply:* I don’t have a reference in itself, but that’s an idea that could help ☝️
*Thread Reply:* This helps.. I'll try this way.. Thanks Marco..
*Thread Reply:* Nifty thing: I have a similar background - I started in backend and services and only started working in JS at all about 3.5 years ago 🙂 Would it be helpful to look at boilerplate projects like this one? Obviously this is highly abstracted for generic browser testing use (and this uses the webdriverio running with cucumber config’d, rather than cucumber runner wrapping wdio as its browser driver), but might help you see how it would be implemented: https://github.com/webdriverio/cucumber-boilerplate
*Thread Reply:* Hi @LayMui I responded in the main #help channel, but I think this might help: https://cucumberbdd.slack.com/archives/C60TKS3SL/p1591394971489400
*Thread Reply:* I would then access the contents of the dataTable as dataTable.name and dataTable.role.
*Thread Reply:* I think my issue is the syntax for Typescript: here is my code snippet: and(/(?:he|she|they) login to the portal/, (table) { const input = table.hashes(); const name = input[0].name;const role = input[0].role; and I got this compilation error: [test:execute] Property 'answeredBy' is missing in type 'RegExp' but required in type 'Expectation<any, unknown>'.
*Thread Reply:* TypeScript is not my specialty. There may be others more familiar with TypeScript that can explain what’s gong on here. I’m not sure which line is throwing the RegExp/Expectation error you’re seeing.
*Thread Reply:* You should probably check the webdriverio gitter channel - the CLI for wdio isn’t the same as that for cucumberjs
*Thread Reply:* yes, this should help you…https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/hooks.md
*Thread Reply:* That's great, thank you! I will track this and try it out when ready
*Thread Reply:* Hi @Abdulaziz - have you considered asking the Selenium/WebDriver community about this? I don't see how this is related to Cucumber.
*Thread Reply:* One option could be store your driver instance in a variable , then have a method to get that instance - getWebdriverInstance() . As Aslak said not a cucumber question
*Thread Reply:* Hello @Richard - I do this to populate a value before each step: ```const { Before } = require('cucumber');
Before(async function (scenario) { this.testname = scenario.pickle.name;
// if we need to grab a screenshot this.shortname = scenario.pickle.name .toLowerCase() .split(' ') .join('-');
});```
*Thread Reply:* Thanks @sonja leaf bit late 🤦♂️:skintone5:. Didn’t see this till now
*Thread Reply:* https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/hooks.md
*Thread Reply:* you can check this out… https://github.com/davidjgoss/cucumber-puppeteer-typescript-starter
*Thread Reply:* be aware of the cucumber cypress option about global step definitions being turned off by default
*Thread Reply:* Please give us more details of what you are trying to achieve.
*Thread Reply:* I have function that take array of objects and reformat its content
*Thread Reply:* @seb I solved it
for object, use rowsHash function to get object structure from data table in step
for array, use hashes function to get array of objects from data table in step
*Thread Reply:* ```$ mkdir shouty [utahcon@beast utahcon]$ cd shouty/ [utahcon@beast shouty]$ ls -al total 8 drwxrwxr-x. 2 utahcon utahcon 4096 Aug 20 13:08 . drwxrwxr-x. 27 utahcon utahcon 4096 Aug 20 13:08 .. [utahcon@beast shouty]$ npm init -y Wrote to /home/utahcon/Projects/src/github.com/utahcon/shouty/package.json:
{ "name": "shouty", "version": "1.0.0", "description": "", "main": "index.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, "keywords": [], "author": "", "license": "ISC" }
$ npm install -D cucumber npm notice created a lockfile as package-lock.json. You should commit this file. npm WARN shouty@1.0.0 No description npm WARN shouty@1.0.0 No repository field.
audited 77 packages in 2.507s found 0 vulnerabilities
$ npm --versbose install -D cucumber npm WARN shouty@1.0.0 No description npm WARN shouty@1.0.0 No repository field.
audited 77 packages in 2.196s found 0 vulnerabilities
$ npm --verbose install -D cucumber npm info it worked if it ends with ok npm verb cli [ npm verb cli '/usr/bin/node', npm verb cli '/usr/local/bin/npm', npm verb cli '--verbose', npm verb cli 'install', npm verb cli '-D', npm verb cli 'cucumber' npm verb cli ] npm info using npm@6.14.8 npm info using node@v12.18.3 npm verb npm-session 1cae8be88e4a4053 npm http fetch GET 304 https://registry.npmjs.org/cucumber 190ms (from cache) npm timing stage:loadCurrentTree Completed in 260ms npm timing stage:loadIdealTree:cloneCurrentTree Completed in 0ms npm timing stage:loadIdealTree:loadShrinkwrap Completed in 97ms npm http fetch GET 304 https://registry.npmjs.org/assertion-error-formatter 76ms (from cache) npm http fetch GET 200 https://registry.npm ... npm timing stage:loadIdealTree:loadAllDepsIntoIdealTree Completed in 1701ms npm timing stage:loadIdealTree Completed in 1835ms npm timing stage:generateActionsToTake Completed in 15ms npm verb correctMkdir /home/utahcon/.npm/locks correctMkdir not in flight; initializing npm verb lock using /home/utahcon/.npm/locks/staging-5cbe75a208033ad6.lock for /home/utahcon/Projects/src/github.com/utahcon/shouty/nodemodules/.staging npm verb unlock done using /home/utahcon/.npm/locks/staging-5cbe75a208033ad6.lock for /home/utahcon/Projects/src/github.com/utahcon/shouty/node_modules/.staging npm timing stage:executeActions Completed in 15ms npm timing stage:rollbackFailedOptional Completed in 1ms npm timing stage:runTopLevelLifecycles Completed in 2179ms npm verb saving [ { name: 'cucumber', spec: '^6.0.5', save: 'devDependencies' } ] npm verb shrinkwrap skipping write for package.json because there were no changes. npm info lifecycle undefined~preshrinkwrap: undefined npm info lifecycle shouty@1.0.0~shrinkwrap: shouty@1.0.0 npm verb shrinkwrap skipping write for package-lock.json because there were no changes. npm info lifecycle shouty@1.0.0~postshrinkwrap: shouty@1.0.0 npm WARN shouty@1.0.0 No description npm WARN shouty@1.0.0 No repository field.
npm timing audit submit Completed in 228ms npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/audits/quick 229ms npm timing audit body Completed in 1ms audited 77 packages in 2.398s found 0 vulnerabilities
npm verb exit [ 0, true ] npm timing npm Completed in 2999ms npm info ok
$ ls -al total 36 drwxrwxr-x. 2 utahcon utahcon 4096 Aug 20 13:09 . drwxrwxr-x. 27 utahcon utahcon 4096 Aug 20 13:08 .. -rw-rw-r--. 1 utahcon utahcon 273 Aug 20 13:09 package.json -rw-rw-r--. 1 utahcon utahcon 23232 Aug 20 13:09 package-lock.json```
*Thread Reply:* I have no idea @Utahcon - it sounds like it's related to your node and/or npm installation. I've followed the same steps as you and ls -al node_modules | wc -l reports 79 for me.
*Thread Reply:* There isn’t - attachments have no naming, it’s just their content and their media type
*Thread Reply:* Hi, I haven’t studied your example in detail, but we have recently added a programmatic usage of the cli to the project here https://github.com/cucumber/cucumber-js/blob/master/compatibility/cck_spec.ts - lots of that code is related to the testing/assertions it’s doing but hopefully it helps
*Thread Reply:* Also, not directly related, but is normal for a world to be constructed multiple times for one scenario?
Using cucumberjs 6.0.5 by the way
*Thread Reply:* Yes you can access the World from an After hook (but not AfterAll). It’ll be the value of ‘this’ (make sure you don’t use an arrow function)
*Thread Reply:* You should expect one stance of World to be created for every scenario that runs (for a scenario outline, that’s one per example)
*Thread Reply:* yes, I found that, thanks 🙂
I did not dig into it, but I put a logging line in my custom world's constructor, and I saw 3 instances of it created
my feature only specifies 1 scenario (no outline being used)
*Thread Reply:* didn't harm anything for me, so I moved on. After(...) is only executed once
*Thread Reply:* Okay, are you able to put an example up somewhere we can reproduce that?
Also just a thought, are you using the retry feature?
*Thread Reply:* not using the retry feature. I can probably get an example up sometime next week 🙂 too late in the day on a Friday
*Thread Reply:* Looks like it is a js equivalent of "array index out of bounds"? https://stackoverflow.com/questions/24911859/cannot-read-property-indexof-of-undefined
*Thread Reply:* This is my scenario outline: ```Scenario Outline: When I make a booking with <card> card Then the booking is created
Examples: | card | | Visa | ```
*Thread Reply:* I made a PR to the wdio repo that fixes this but I am surprised it doesn't appear anyone else has run into it.
*Thread Reply:* For anyone mildy interested, I think what happened is on a resource poor system, when a couple of the tests failed because of a timeout, either selenium or protractor crashed, which fails the rest of the tests.
*Thread Reply:* I know you posted this a long time ago, but the times I have seen that, it usually indicates some sort of race condition. Let's say, you forgot an await on something, and 95% of the time it executes just in time, but on those 5% of times it doesn't and browser session is killed, it will throw that.
*Thread Reply:* Thanks @Kārlis Amoliņš I upgraded typescript and tslint to the latest and made use of the new newer lint rules to find the missing awaits, looks like its stable now. Cheers
Hi, BeforeAll hooks don’t have access to a world because a world belongs to a scenario, and BeforeAll happens before any scenarios. There is some discussion on this issue https://github.com/cucumber/cucumber-js/issues/1393 including a possible way forward
*Thread Reply:* Yes, I know. I read this discussion. Hovewere I was thinking how to make this possible. I was trying with call() method to share contex from customWorld but without luck.
*Thread Reply:* There is a short migration guide here: https://github.com/cucumber/cucumber-js/blob/master/CHANGELOG.md#700-rc0-2020-09-14
*Thread Reply:* Does @cucumber/cucumber play nice with @wdio ? Yes. Cucumber plays nice with anything you put inside your step definitions or hooks, because it is blissfully ignorant of what your code does. It just reports errors if your code throws errors.
*Thread Reply:* Ok I guess I'll have to see since I'm not exactly sure why when I was trying to run cucumber for my current test framework (which has a different dependency structure than a past one where I used webdriverio and launched things with cucumber, it was reporting 0 tests, vs running wdio commands. Perhaps I was using it incorrectly.
*Thread Reply:* I'll have to investigate. Thanks for sharing the migration guide!
*Thread Reply:* There are a few options. You can set it at configuration - when you pass the step defs in, just pass in the ones you want to use. You can also modify the step definitions themselves to have a conditional function that determines which behavior should be leveraged. However, the best practice is for your language to be expressive. Having ambiguous language that could indicate multiple different behaviors tends to bite you in the ass later.
*Thread Reply:* You should only need to write the step definition once. You can then re-use it in multiple tests. If you are attempting to write step definitions that have the same name but perform different actions, consider changing one of them to be more specific.
*Thread Reply:* @Cat - test manager wrote a large number of test cases/scenarios and she expects that it will be a big overload to rewrite them in a unique way. Since Cucumber is well known and widely used, I had a hope that there is a better and less time consuming solution.
*Thread Reply:* @Mona Ghassemi - I am not sure I follow your line of thoughts completely. We use Jenkins and Xray for Jira. We will have to have multiple feature files in one repository branch to be able to do testing and get report back to Xray. Let's say we have 100 test scenarios and 50% have steps which have to be repeated. The idea of rewriting the test scenarios is pushing our manager to get rid of Cucumber. Question - how to save Cucumber? 😄
*Thread Reply:* They don’t all need to be unique. Quite the opposite. They can be paramaterized in such a way that they can be quite flexible. It’s a time investment, but all test automation is (and should be). If she isn’t anticipating that, she got into the wrong line of business
*Thread Reply:* If it would be up to me - I would modify the test scenarios a bit. I even did add one extra word in the second scenario ("the user" instead of "user") to show my manager that we do not have to rewrite a lot. But she did not like it. She think that it will take a lot of maintenance to continue the project. 🧐 I disagree but ...
*Thread Reply:* @Cat - thanks for the input. I thought it was my duty to check if we can do something else on the test engineers side.
*Thread Reply:* @aberdan I don't think adding extra words like "the" is of any help. Cat's comments on abstraction and parametrization are on point.
*Thread Reply:* Part of the reason that you don’t want to have step defs that are too similar or identical is because it then becomes very difficult to determine what they’re doing. Look instead at the step definitions to see why they’re different. Are they actually that different? You may find that they aren’t, or you may find that some added conditional handling and state control will go a long way.
*Thread Reply:* The purpose of the scenarios is to express the behavior. If you have 5 different step definitions that do different things, but are verbally expressed the same way, that makes it harder to detangle the behavior, not easier, which defeats the point.
*Thread Reply:* If I say Given the user has logged in and one of my step defs logs into a completely different application, I’m screwing myself over. Even if it’s just a differently permissioned user or there are slightly different UI flows involved, it doesn’t necessarily express the state and, as a result, introduces confusion rather than eliminating it
*Thread Reply:* So given the situation, I’d consider timeboxing it to see how many step defs you have for a given thing and what they actually do. Are they doing the same thing, effectively? Do they maybe just have slightly different selectors? Can they be condensed to a single step def? That’s going to help you champion cucumber better if you can show the streamlined step definitions and expressive feature files
*Thread Reply:* As an example (terms and conditions page has two check boxes - for privacy policy and for terms and conditions): Scenario 1: Test accept privacy policy only Given user has activated account Then user navigates to terms and conditions page where terms and conditions can be reviewed And user can click the terms and conditions side bar to scroll down to the bottom of the page where check boxes are visible When user accepts only the privacy policy And clicks the Register button Then user cannot progress to the next page When user accept terms and conditions and privacy policy And then clicks the Register button again Then user is taken to the Utility Page
Scenario 2: Test accept terms and conditions only Given user has activated account Then user navigates to terms and conditions page where terms and conditions can be reviewed And user can click the terms and conditions side bar to scroll down to the bottom of the page where check boxes are visible When user accepts only the terms and conditions And clicks the Register button Then user cannot progress to the next page When user accept terms and conditions and privacy policy And then clicks the Register button again Then user is taken to the Utility Page
As you can see, only step #4 is different, all other steps are the same. And it is just one example, we have much more in our backlog in Jira. I do not write the Cucumber scenarios, I write JavaScript code for automated testing and we have no problems with JS. What should we do? Does the test manager have to write one Scenario for steps 1-3 and another scenario for steps 5-9? Which then can be reused when needed?
*Thread Reply:* ok so, firstly: you’re testing something twice. Imo, you need to have a distinct scenario for the positive case (acceptance of both policies). Then there’s a separate scenario for each of the negative conditions (having only one of the policies checked). Secondly: there are a few ways you can streamline this to be more elegant and have more DRY code. For example: All you have to say is “When user accepts <policy>” That’s a single step def that takes an arg. That’s all it has to be. Just have it take a parameter and, based on what you pass in, it clicks whichever checkbox. Similarly: “And then clicks the Register button again” - this is unnecessarily redundant. Just say “And clicks the Register button” - it’s the same action, no matter what. If it’s the same action, use the same words.
*Thread Reply:* These kinds of things will help make the code and the features less repetitive. It’ll become faster over time.
*Thread Reply:* It also helps you spot areas where the code can be abstracted to make it more efficient
*Thread Reply:* @Cat - a personal huge Thank you! Everything you said makes a lot of sense to me. I will try to convince my manager to restructure the test cases in a less redundant way. I hope she will listen. 🙂
*Thread Reply:* I figured it out - these lines in node-assertion-error-formatter cause me to lose my assertion message because my message starts with "expected"
*Thread Reply:* node v12.16.1
shell
ReferenceError: shell is not defined
at repl:1:1
at Script.runInContext (vm.js:130:18)
at REPLServer.defaultEval (repl.js:435:29)
at bound (domain.js:427:14)
at REPLServer.runBound [as eval] (domain.js:440:12)
at <a href="http://REPLServer.onLine">REPLServer.onLine</a> (repl.js:760:10)
at REPLServer.emit (events.js:315:20)
at REPLServer.EventEmitter.emit (domain.js:483:12)
at REPLServer.Interface._onLine (readline.js:329:10)
npm run test-cucumber
npm should be run outside of the node repl, in your normal shell.
(Press Control-D to exit.)
npm run test-cucumber
npm should be run outside of the node repl, in your normal shell.
(Press Control-D to exit.)
*Thread Reply:* thanks for replying Mona. If you wait for the editor to load then press F1 while in one of the files, then type shell you should see this
*Thread Reply:* but I figured out the reason it was happening and posted it here ... https://github.com/charlierudolph/node-assertion-error-formatter/issues/3
*Thread Reply:* we get TypeError: Cannot read property 'address' of undefined
*Thread Reply:* You should be able to use babel. (I assume you can also use webpack, but I’m personally more familiar w/babel).
*Thread Reply:* if we change it to module.exports = app cucumber works, but then our app doesn't build
*Thread Reply:* Right. Natively, cucumber isn’t going to be able to leverage commonjs exports. You have to configure that explicitly. It’s distinct from your application transpile.
*Thread Reply:* This discussion gives some high level instructions.
As mentioned, you’ll want to configure something to the effect of "presets": [["env",{"modules":"commonjs"}]]
You may need to fiddle around with it a bit. This was ~2 years ago, looks like, so some of the verbiage could have changed since then. When in doubt, check the babel docs.
*Thread Reply:* Ok. Without knowing how you’ve configured it, how you’re running it, what commands you’re using it, or anything about what you’re trying to run, there’s very little I can do to help other than point you in the right direction. 🤷
*Thread Reply:* That’s going to be an angular question - not a cucumber question.
@Kyle Bailey the JSON formatter is deprecated: • https://github.com/cucumber/cucumber-js/blob/master/CHANGELOG.md#breaking-changes • https://github.com/cucumber/cucumber/blob/master/messages/README.md
*Thread Reply:* Yes! Provided I have configured a parameter as shown below.
defineParameterType({
name: "foobar",
regexp: /foo|bar/
});
Furthermore I have a step definition like
When("I put a {foobar} in my inventory", (fooOrBar) => { ... });
My understanding is that cucumber-expression will compile an expression, but if the resulting expression is a mere interpolation EG. /^I put a foo|bar in my inventory$/ , then that will probably have unexpected results
*Thread Reply:* The expression will match these steps:
I put a foo in my inventory
I put a bar in my inventory
Anything else will not match. What do you mean by interpolation? What kind of unexpected results are you anticipating?
*Thread Reply:* I'm mostly curious at this point about the workings of cucumber-expressions and I'm not experiencing anything out of the ordinary or what you describe
*Thread Reply:* Am I correct to assume that using anchors for start / end of string will mess with it?
*Thread Reply:* Yes. Regular Expressions is a language with its own syntax rules. Cucumber Expression is also a language with its own syntax rules. You can't mix them. It's like Russian and Spanish.
*Thread Reply:* Of course - I just wanted to give you a heads up since we plan to remove the JSON formatter from future versions of Cucumber.
Have you seen this? https://github.com/cucumber/cucumber-js/blob/master/docs/custom_formatters.md
^^ @Kyle Bailey
*Thread Reply:* Ah, gotcha!
I have, and I made a custom formatter that does what I want, but I'm uncertain how to let that formatter run in parallel to the standard formatter
*Thread Reply:* @Kyle Bailey you can specify multiple formatters on the command line. Cucumber will send events to all of them. You shouldn't need to worry about what order they run in.
*Thread Reply:* @Adam Merritt I believe it was removed because we changed the internal data structures as events were refactored to use cucumber-messages: https://github.com/cucumber/cucumber-js/blob/master/CHANGELOG.md#breaking-changes
What do you need scenario.result.exception for? Perhaps we can suggest an alternative approach?
*Thread Reply:* I'm using cucumber for UI automation testing and have an error type that will store the Dom structure as JSON when it throws. The problem is I want that piece (the UI interface) decoupled from Cucumber, so it wouldn't have access to the World
*Thread Reply:* If I use "Set definition function wrapper " then it's dependent on no other component overriding it
*Thread Reply:* it's already there https://github.com/kozhevnikov/cucumber-pretty/issues/14 🙂
*Thread Reply:* 9 years ago I discovered that prebuilt step definitions are a bad idea: https://aslakhellesoy.com/post/11055981222/the-training-wheels-came-off
*Thread Reply:* I will read! I suspect I have a different use case than is normally in here.
*Thread Reply:* I really enjoyed that. Well written article and it brings up a lot of good points and makes me question if cucumber really is the solution we’re looking for. Do you have bandwidth for a full discussion in here, @Aslak Hellesøy?
*Thread Reply:* The point I’m trying to make in that article is that scenarios should reflect domain language and business rules. Not the user interface.
*Thread Reply:* It makes sense, and that’s why maybe another tool is better for us than cucumber - we’re testing interface, not domain/business rules.
*Thread Reply:* One of the problems is that we need humans to be able to write them.
*Thread Reply:* We need something that lets us write human sentences to test forms for correct behavior.
*Thread Reply:* So you may well have more success using a different tool. Related articles: https://cucumber.io/blog/collaboration/the-worlds-most-misunderstood-collaboration-tool/ and https://cucumber.io/blog/bdd/bdd-is-not-test-automation/
*Thread Reply:* You can do that with Cucumber, but your tests will be brittle, slow and hard to maintain
*Thread Reply:* Is there something else we can do that with that would be better?
*Thread Reply:* We’re basically doing regression testing, we just need humans to be able to write it, not coders.
*Thread Reply:* We’re working on putting out emergency court forms like stopping evictions and requesting help with domestic violence. Those forms have certain things about them that absolutely have to work for them to be safe for people to use them.
*Thread Reply:* We don’t have a ton of coders. We do have volunteers, most of them with legal experience (which doesn’t help much).
*Thread Reply:* Some tests aren’t just ‘click this button’ and ‘type this in’. We’re building a framework so that these can be more easily created and deployed. That is, all the pages that ask for names will be the same. That can let us build a step that fills in a name on a name page. We haven’t figured out how to abstract more than that without every form maker having to write code that they don’t have time to learn.
*Thread Reply:* I’d recommend trying out a declarative style rather than imperative. Ben Mabey’s original blog post is gone I think, but here is a riff on it: http://blog.nickcox.me/2012/12/17/refactoring-cucumber/
*Thread Reply:* We also want people to be able to re-use the tests we write, of course, which is why I’m trying to make a package.
*Thread Reply:* Note that a declarative approach is incompatible with pre-built step definitions, which tend to be imperative.
*Thread Reply:* I really recommend that step definitions are written, maintained and refactored by the same people who build the application.
*Thread Reply:* We’re teaching the volunteers to code, but they’re coding in python in a framework that simplifies a lot of what they do. They’re not ready to write files like “#spec/acceptance/forms/feedback_steps.rb”
*Thread Reply:* I understand that in an ideal world we’d have the money to hire more coders, but we don’t have that.
*Thread Reply:* I understand. I think you should try what you’re planning. It might work out, but just wanted to warn you of some common pitfalls.
*Thread Reply:* For sure, and these are really good points. I’m going to bring them up with the team and see what people think we can do about these. I can’t currently think how to account for them within our budget, but maybe someone else will have an idea.
So far the test writing has gone pretty well, but we’re copying the step code into every repo. No one can take advantage of updates very easily. The volunteers are not great at using github and it’s taking a lot of developer time to update the steps.js files, etc.
*Thread Reply:* Right now I’ve set aside time to learn how to make a package that can take care of some abstractions. No pressure, but is that something you have knowledge about and a discussion you have bandwidth for, @Aslak Hellesøy? I understand if you feel I’m just wasting your time. I can try to find answers somewhere else.
*Thread Reply:* We need to figure out something that can take pressure off of testing by hand. We have some forms that can take 30 - 45 min. to fill out [accurately by a human] (like the domestic abuse one).
*Thread Reply:* I'm afraid I don't have much time to help. I recommend you start an issue on GitHub with as much detail as possible. What you're trying to do might even require a change to Cucumber.js - I'm not sure.
*Thread Reply:* Ok, thanks. Do you have a sec to give me tips on how to improve my question so that it provides better context or is more clear?
*Thread Reply:* Also, just wanted to say I think it's great Cucumber is used for what you're working on!
*Thread Reply:* Thanks! So far it really has saved us a ton of time and let us re-focus on the users. Just need to make the ecosystem easier to expand on.
*Thread Reply:* This is the package: https://github.com/plocket/docassemble-cucumber
*Thread Reply:* So I as a developer would use this package?
And one of the main issues is not having env vars or config?
*Thread Reply:* I prefer to use https://www.npmjs.com/package/config over dotenv in a situation like this
*Thread Reply:* Let me pull it down and see if I can use it as a consumer
*Thread Reply:* Thanks. I’ll take a look at config. I’m still not sure how to get it so the developers don’t have to write any .js files, just .feature files.
*Thread Reply:* so i should be able to do this right?
const Framework = require("docassemble-cucumber");
console.log("Framework", Framework);
*Thread Reply:* I’m not sure you should be able to log anything, you just need to be able to use a .features file with it.
*Thread Reply:* Only opening the interview will work for you if you’re not visiting the right sight, though, so you might only be able to ‘start an interview at’
*Thread Reply:* well i assume i'd need a feature file specific to your tests cases ya?
*Thread Reply:* y’know what, there’s no reason you shouldn’t be able to run it with the test repo. Just a sec.
*Thread Reply:* Use the npmjs branch and don’t bother trying to do anything other than running the tests.
*Thread Reply:* Can’t be because the file’s not able to use them anymore.
*Thread Reply:* Just going to double check - you’re using the npmjs branch?
*Thread Reply:* ran
npm i
npm link docassemble-cucumber
npm run cucumber
*Thread Reply:* You at least confirmed it works on someone else’s environment. Other people are taking the weekend off for some crazy reason.
*Thread Reply:* Trying to get rid of as many files as I can to simplify the process for folks. I wanted to reduce the number of folders and put everything in features, but someone brought up our desire for unit tests in the future.
*Thread Reply:* Ideally developers would only have to write .feature files and there’d be as little setup as possible.
*Thread Reply:* wait whats diff between the main package and the published one?
*Thread Reply:* Yeah, except it doesn’t. I haven’t been able to get it to.
*Thread Reply:* I can see that providing ~the~ [that] link was misleading. Learning a lot about communication today.
*Thread Reply:* where is this?
Error: Cannot find module './interview-constants.js'
*Thread Reply:* Yeah, trying to get rid of those env vars, but you shouldn’t be trying to trigger them anyway for this part.
*Thread Reply:* you should probably package cucumber with your package
*Thread Reply:* I got recommended to only include it as a peer dependency. I guess I’ll put one vote for each so far, lol.
*Thread Reply:* before you do it all. do a npm i -g . inside your package
*Thread Reply:* i'm still running in ~/source/repos/docassemble-IntegratedTesting
*Thread Reply:* I’m a bit distracted with some kind of …memory leak? in my computer right now that makes it think my folders have a ton of files in them.
*Thread Reply:* i'm not sure how to tell cucumber where to look for the stesps
*Thread Reply:* I may not have to move the steps if I do what your code just did. I’ll have to try it after I figure out how to keep my computer from exploding.
*Thread Reply:* If you’re ever back on, @Jeremy Lorino, I’m trying to replicate your file and I’m not sure where to require the files for the package - scope.js, world.js, etc. Where does that happen?
*Thread Reply:* Other than that, I’m also curious about what’s going on there, but I can probably look that up myself.
*Thread Reply:* I think I may be using it closer to what was intended now and I’m getting:
Command failed: sh stuff/node_modules/.bin/cucumber-js tests/features/interactive.feature tests/features/observations.feature
stuff/node_modules/.bin/cucumber-js: line 3: syntax error near unexpected token `'../lib/cli/run.js''
stuff/node_modules/.bin/cucumber-js: line 3: `require('../lib/cli/run.js').default();'
*Thread Reply:* All my file did was pass through the list of files to cucumber-js
It requires a list of space delimited files
*Thread Reply:* So it needs the files from the package as well? [I’ll just try it out]
*Thread Reply:* I would have to see the code. Not at my computer right now.
Push a branch and link it.
*Thread Reply:* It’s pushed to main. I can always revert to an old commit at v2.
*Thread Reply:* Thanks for all your help. A friend had some time and we figured out I just need to add the main file as required in the package script just before naming the path to the features. This was the line: https://gist.github.com/plocket/55c0079784265ca2a06630cf489f20da
*Thread Reply:* I see you have --format usage set. I think this will show you which step definitions are used. Adding -b shouldn’t have any effect.
Are you seeing a table of steps or pass/fail reports?
*Thread Reply:* It's not the best format but it's the best I could find. I think the format I am used to was cucumber-pretty but I realized it's no longer compatible with the new package.
*Thread Reply:* @Mona Ghassemi --format usage doesn’t execute your scenarios so it won’t tell you if it is passing or failing. I think --format progress is what you are looking for.
*Thread Reply:* there is a pretty formatter available, but unfortunately it doesn’t suppor the latest version of cucumber https://github.com/kozhevnikov/cucumber-pretty/issues/14
*Thread Reply:* yes, I was asking about this not long ago and then I read the Github discussion 😞
*Thread Reply:* I don't like --format progress; it doesn't show my steps:
```..
1 scenario (1 passed) 2 steps (2 passed) 0m00.307s (executing steps: 0m00.303s)```
*Thread Reply:* Oh I see usage is going to be weird once I add more scenarios...
*Thread Reply:* yes --format progress will only show the steps on failed scenarios (if I remember correctly). But --format usage doesn’t actually run your scenarios so you won’t know if anything passes or fails.
Hello, I'm trying to define cucumber parameters in a file as in this example: https://stackoverflow.com/questions/60758671/how-do-i-use-hooks-before-after-with-cucumber-js-tsflow
*Thread Reply:* This is the code that line refers to: ```import setWorldConstructor from '@cucumber/cucumber' import setEnv from "../config/setEnv";
function CustomWorld() { this.ENV = setEnv() console.log(this.ENV) }
setWorldConstructor(CustomWorld) // <-- this is line 9```
*Thread Reply:* @Mona Ghassemi your import statement is incorrect. Try import { setWorldConstructor } from ‘@cucumber/cucumber’
*Thread Reply:* Also I believe I found the documentation but it wasn't helping with this error; I was a bit thrown off 🙂
*Thread Reply:* https://github.com/cucumber/cucumber-js/blob/master/features/profiles.feature for future reference in case anyone is looking at this
Hi - (Working with major 7) I'm getting type error when returning a string 'skipped' , but the docs still say I can do this
[12:45 PM] is this a bug with the type? [12:46 PM] https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/step_definitions.md#skipped-steps
*Thread Reply:* @Adam Merritt can you please share the full stack trace?
*Thread Reply:* @Balazs Mandel add an id or class to it so you can identify it more easily. Or use XPath.
*Thread Reply:* @Aslak Hellesøy yeah that was the solution thanks !!!
*Thread Reply:* That sounds like more of a webdriverIO question, not a cucumber question, but glad you got your answer!
*Thread Reply:* The error is that one: ``` scenarios (7 passed) 23 steps (23 passed) 0m20.580s (executing steps: 0m20.548s) events.js:109 throw new ERRINVALIDARG_TYPE('listener', 'Function', listener); ^
TypeError [ERRINVALIDARGTYPE]: The "listener" argument must be of type function. Received an instance of Object at checkListener (events.js:109:11) at ClientRequest.once (events.js:431:3) at new ClientRequest (httpclient.js:194:10) at Object.request (https.js:314:10) at Object.request (/home/florent/app/nodemodules/agent-base/patch-core.js:25:22) { code: 'ERRINVALIDARG_TYPE', CallSite {}, CallSite {}, CallSite {}, CallSite {}, CallSite {}, CallSite {}, CallSite {}, CallSite {}, CallSite {}, CallSite {} ], } npm ERR! code ELIFECYCLE npm ERR! errno 1```
And the file I commented is patch-core in agent-base module
```'use strict'; const url = require('url'); const https = require('https');
/**
* This currently needs to be applied to all Node.js versions
* in order to determine if the req is an HTTP or HTTPS request.
*
* There is currently no PR attempting to move this property upstream.
**/
// const patchMarker = "agentbasehttpsrequestpatched";
// if (!https.request[patchMarker]) {
// https.request = (function(request) {
// return function(options, cb) {
// let options;
// if (typeof _options === 'string') {
// options = url.parse(options);
// } else {
// options = Object.assign({}, _options);
// }
// if (null == options.port) {
// options.port = 443;
// }
// options.secureEndpoint = true;
// return request.call(https, options, cb);
// };
// })(https.request);
// https.request[patchMarker] = true;
// }
// /**
// * This is needed for Node.js >= 9.0.0 to make sure https.get() uses the
// * patched https.request().
// *
// * Ref: https://github.com/nodejs/node/commit/5118f31
// **/
// https.get = function (url, _options, cb) {
// let options;
// if (typeof _url === 'string' && _options && typeof _options !== 'function') {
// options = Object.assign({}, url.parse(url), options);
// } else if (!options && !cb) {
// options = _url;
// } else if (!cb) {
// options = _url;
// cb = _options;
// }
// const req = https.request(options, cb); // req.end(); // return req; // };```
*Thread Reply:* could you create an issue on cucumber-js with the info / file ref and a reproducible scenario.
*Thread Reply:* ok, I'll try to find a minimal reproducible example
*Thread Reply:* @luke I find a way to reproduce this with a minimal amount of code: clone: https://github.com/Florent-Bouisset/testCucumberReports.git npm install, and then run npm run cucumber => it should work then npm run cucumberReport => there is an error and the report is not published
*Thread Reply:* can you push that up to the issue tracker with the relevant links and we can take a look.
*Thread Reply:* @luke before creating an issue I look if there was a problem with the tool puppeteer that I am using for doing e2e test, and I found out that if I upgrade puppeteer to the last version, I don't have the pblm anymore, so it's solved 😛 thx
*Thread Reply:* Yes that's what I would have suggested first when you are getting such issues.
*Thread Reply:* yes, they were not up to date because the new updates introduces breaking change that broke some parts of the app..
*Thread Reply:* Yeah breaking changes are a whole project to fix 🙂
*Thread Reply:* So is this still a problem after upgrading @Florent Bouisset ?
I'm scratching my head here... How can null be compatible with ReadonlyArray?
*Thread Reply:* To be more clear, I have a feature file like the following:
```Feature: Calculator
Scenario: Test Given I have number <n1> When I sum with <n2> or multiply by <n3> Then Result should be <result>
Examples:
| n1 | n2 | n3 | result |
| 1 | 2 | 3 | 3 |
| 2 | 4 | 3 | 6 | ```
*Thread Reply:* when I run cucumber-js --format snippets the result is two identical snippet for each sentence (because of the two executed scenarios)
*Thread Reply:* It should be: Scenario Outline: not Scenario:
Let me know if that helps.
Ref: https://cucumber.io/docs/gherkin/reference/
*Thread Reply:* I've already tried with Scenario Outline.. Same behaviour
*Thread Reply:* Hm, I can't really play around with it. I think Scenario Outlines are broken in that RC or at least, I can't get it to work as it doesn't recognise the Outline keyword.
But even on Master branch I am having issues with Scenario Outlines - though on master I get a different issue - the steps are not being recognised. 😞
*Thread Reply:* Thanks @Kārlis Amoliņš.. so, should I create an issue on github?
*Thread Reply:* Based on what someone else wrote - Outlines should be working so trying to make it work in a fresh repo, will let you know if that works - and if it does, will try to repro the issue you saw
*Thread Reply:* I think I was somewhat off (misremembered what snippets does.
I think the behaviour you see is somewhat right. snippets print out definition for each undefined step.
So from the perspective of Cucumber it is doing what it should even though there is no practical benefit to it.
*Thread Reply:* Yes, but maybe for the snippet generation purpose it would be very useful to produce just a single snippet per sentence
*Thread Reply:* @Kārlis Amoliņš Scenario is interchangeable with Scenario Outline since a while now...
*Thread Reply:* Not sure why you think Scenario Outline is broken?
*Thread Reply:* @Aslak Hellesøy Hmmm, interesting.. Not sure why I was so convinced about them not being interchangeable. Thanks for pointing that out and apologies for spreading misinformation!
And I managed to get Scenario Outlines to work at a later point, I think my environment was messed up. 😓
*Thread Reply:* Hello. You should probably ask your question in <#C5YHPPJMP|help-cucumber-jvm> since it is Java specific
*Thread Reply:* Somewhat related question - are you able to run Scenario Outlines at all when using the RC?
*Thread Reply:* I'm not familiar with "RC"? But i'm currently able to run them.
*Thread Reply:* Apologies, by RC I meant Release Candidate (which is the latest released version of @cucumber/cucumber) and my question was, are you able to run Scenario outlines when using latest version of @cucumber/cucumber ?
*Thread Reply:* I tried to assist someone earlier today/yesterday, but I couldn't get Scenario Outlines to work at all when using the latest version of cucumber 7.0.0-rc.0
*Thread Reply:* yes, it is running the scenario outlines just fine.
Of course I needed to modify my step_defs to require @cucumber/cucumber instead of just cucumber
*Thread Reply:* the only problem is as stated above, it seems to have forgotten how to find overloaded functions
*Thread Reply:* Out of curiosity, do you still have @types/cucumber in your repo?
*Thread Reply:* Yeah, I think I know what this is.
When you npm link your library, you introduce a problem where your libary and the host project are using different instances of cucumber (each from their own node_modules dir) which causes failure.
There is a bit of info and some suggestions here https://github.com/cucumber/cucumber-js/issues/1146. My solution on a few projects has been to use the "import-cwd" package (https://www.npmjs.com/package/import-cwd), like this:
const importCwd = require("import-cwd");
const {Given, When, Then} = importCwd("cucumber");
That way even when linked you're always using the host project's cucumber-js instance and all should be well. Hope this helps.
When I use puppeteer to download a file, cucumber does not finish, it just hangs. I found this way to detect about when the the file will be downloaded, I don’t always want to close the browser when the download completes, I just want cucumber to be able to finish at the end. What are some resources that can point me in the right direction to solve this?
*Thread Reply:* I'm just curious what packages are you using to run cucumber with ts? Last time I checked, I saw the ts cucumber package (not sure exactly what it was called but I can check) was still using old cucumber package (prior to the re-name) so I was not able to transition to ts at that time. I'd appreciate if you could share your package.json set-up.
*Thread Reply:* Hi, sure, I'am using:
"@types/cucumber": "6.0.1",
"cucumber": "6.0.5",
"ts-node": "9.0.0",
my tsconfig is the following:
{
"compilerOptions": {
"target": "ES2019",
"module": "commonjs",
"declaration": true,
"sourceMap": true,
"strict": true,
"strictBindCallApply": false,
"resolveJsonModule": true,
"esModuleInterop": true,
"allowSyntheticDefaultImports": true,
"noUnusedLocals": false,
"noUnusedParameters": false,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"strictPropertyInitialization": false
}
}
and the script I use to execute cucumber is
npx cucumber-js --require-module ts-node/register --require 'features/****/**.ts' features/****/**.feature
*Thread Reply:* Ah, looks like indeed you are using a prior version of cucumber. We're using "@cucumber/cucumber": "^7.0.0-rc.0". Was excited about the reporting feature and this was a new project. I was first attempting the setup using this guide: https://dev.to/denolfe/cucumber-js-with-typescript-44cg and I believe it was indeed @types/cucumber that was using the old dependency, and it seems it has not been updated recently: https://www.npmjs.com/package/@types/cucumber 😞
*Thread Reply:* But I've also heard typescript is a lot of overhead as well so 🤷
Hey @Roberto Zen
I think you’re pretty close; there’s a couple of different ways you could solve it. When you add this: any as the first parameter to a function, it doesn’t actually end up being a parameter in the JS, it’s just compiler advice. So the structure of foo comes out as per your first example, with just one parameter. If you want to control what the value of this is within the function, you could change the way you call it to use Function.prototype.call , like this:
await foo.call(this, val1);
Alternatively, you could leave your usage as it is, and make the world a normal parameter in your function - so rewrite it to something like:
function foo(world: any, val: string): Promise<any> {
const value = `${world.worldParameter}_${val}_${world.worldParameter}`;
console.log(value);
return Promise.resolve('success');
}
(I would argue that second one is a bit clearer, FWIW.)
Finally, for a different angle again, you could always make your World a class and make foo a method on it, so you can just call it from each step.
Hope this helps!
Opened issue here: https://github.com/cucumber/cucumber-js/issues/1489 around getTestCasesFromFilesystem Thanks for response @David Goss
*Thread Reply:* A SDET that attends a local meetup I attend shared https://github.com/thomaschaplin/webdriverio-template which got me down a few rabbit holes
*Thread Reply:* I am using jest-cucumber but it does not really supporting reporting. They had one version (one year old) which had the reporting included but the maintainer is not happy with it and haven't included it in any other version. Now I am stuck, cannot update and will wake up with security leaks 😞
*Thread Reply:* so the question is if you have an idea how and with what I could write jest tests but have cucumber features and reports?
*Thread Reply:* Hi @Krisztina Hirth, I’d like to understand more about the motivation behind using jest-cucumber. Reading between the lines, it sounds like you aren’t that bothered about writing your specs in Gherkin - you write them in jest - but just want good reporting, is that fair?
With the release of cucumber-js 7.0.0 today, a run of cucumber generates standard messages that can be consumed by a formatter (including our own html formatter). So in theory you (or, the maintainer of jest-cucumber) could generate those messages from jest and use the same formatter - would be a bit of work involved but I could give some pointers.
*Thread Reply:* Happy New Year everybody, sorry @David Goss for the silence, I had holiday and I had to reduce my amount of communication 🙂
*Thread Reply:* After reading your question I realized that it was an error to not start with cucumber-js from the beginning on
*Thread Reply:* I started to rewrite the tests today afternoon and I still would have a couple of questions:
*Thread Reply:* The first one: is there a possibility to configure the call via a config object? My npm command is 2 rows long already 🙄
*Thread Reply:* yep you can define “profiles” in a cucumber.js file in your project, see https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#profiles
*Thread Reply:* simple example: https://github.com/davidjgoss/cucumber-puppeteer-typescript-starter/blob/master/cucumber.js
*Thread Reply:* an example like this would be very helpful in the docs
*Thread Reply:* agreed! i have a few things to do in the docs so will add that
*Thread Reply:* Last question: I read that the JSON reports are deprecated
*Thread Reply:* I tried the current format and it is cool but it can be nicer :)
*Thread Reply:* Do you know the html-cucumber-reporter (or similar)?
*Thread Reply:* so that one and others use the (deprecated) JSON formatter output, is that what you mean?
*Thread Reply:* This is what I use at the moment but it needs json as entry file.
*Thread Reply:* The question is: what is your recommendation to be used instead of json?
*Thread Reply:* if you are not using it is ok, I will check tomorrow
*Thread Reply:* the JSON formatter is deprecated but wont be removed until 8.0.0 which is probably some way off, so it’s fine to use for now. the core team have also written libraries for converting between the JSON format and the new messages protocol, meaning formatters like that should be able to refactor so they can tae the new messages as input without much difficulty before 8.0.0
*Thread Reply:* in the meantime it would be good to get your feedback on the built in html formatter. it’s quite new but now the infrastructure is in place so i think it could be iterated on quite well
*Thread Reply:* I think it is cool, much more informative as the old one 🙂
*Thread Reply:* I am on my private laptop now but tomorrow or the day after I will create some screenshots and then I can show you, what I'm missing if you would like :)
*Thread Reply:* So, I'm back 🙂
I migrated almost all my tests (50 scenarios). Regarding the reports: this is how the html-report looks like (but maybe your know this already, it is shown here https://www.npmjs.com/package/cucumber-html-reporter)
*Thread Reply:* I tried to use the ndjson-file but unfortunately didn't work:
⚠️ 😞 Unable to parse cucumberjs output into json: '%s' ./reports/cucumber.ndjson SyntaxError: ./reports/cucumber.ndjson: Unexpected token { in JSON at position 217
*Thread Reply:* Question 1: I saw that you are using it. What are you doing with it?
*Thread Reply:* The currently generated html report is not bad, as I already mentioned it it contains all the information
*Thread Reply:* Generally there are a few differences: • in the cucumber-html-report the scenarios are displayed, the steps are collapsed, in 2 columns on the page • in the cucumber.js report the files are listed and collapsed if green. This means for me that in this case the report is not intended as a documentation but as a regression testing tool • the cucumber-html report has that nice pie chart showing the overview. This is really nice but it is surely more a nice2have
*Thread Reply:* That's it regarding the reports. the cucumber-html-report is really nice and I will keep it until the json-format still exists but I will switch afterwards to the html-format.
*Thread Reply:* Now to the CustomWorld: I tried it today but it didn't work. When calling this in another file it was an empty object. I tried different filenames but it didn't help.
```[11:46] // file world.js or CustomWorld.js
const { setWorldConstructor } = require('@cucumber/cucumber');
const props = { insuranceOption: null };
class CustomWorld { constructor() { this.variable = props; }
setInsuranceOption(option) {
props.insuranceOption = option === 'add';
this.variable = {
...props
};
}
}
setWorldConstructor(CustomWorld);
// support file When(/^I choose to (add|not add) an additional travel insurance$/, async (option) => { console.log(this); this.setInsuranceOption(option); }); // error: TypeError: this.setInsuranceOption is not a function at CustomWorld.When (/home/``` What do I miss?
*Thread Reply:* > Question 1: I saw that you are using it. What are you doing with it?
You mean what am I doing with the NDJSON? Those messages is used as the input to the built-in HTML formatter (internally the html formatter option just pipes them through directly to save you doing it). Long term the idea is that most/all formatters use those messages as the input, meaning they could be common across all cucumber implementations.
*Thread Reply:* > the cucumber-html report has that nice pie chart showing the overview. This is really nice but it is surely more a nice2have Agree it's a nice to have, although the underlying desire for a ratio/percentage of pass:fail is interesting.
*Thread Reply:* The way you've set your custom world up is fine (though you may want to take the options argument to the constructor and call super(options) with it, so that you still get the attach function etc.
*Thread Reply:* The problem is with the step definition - you have an arrow function there, but it needs to be a regular function literal instead, like:
When(/^I choose to (add|not add) an additional travel insurance$/, async function(option) {
console.log(this);
this.setInsuranceOption(option);
});
(More about why here https://github.com/cucumber/cucumber-js/blob/master/docs/faq.md#the-world-instance-isnt-available-in-my-hooks-or-step-definitions.)
*Thread Reply:* I finished the migration and I am very satisfied (I ask myself why I even started with the jest-cucumber package 🤔 ) The custom world works very well too. Thanks a lot for your help!!
One last feedback to get an easier usability of the lib: I think a small hint or a few examples how to write the tests with chai.js would be helpful.
*Thread Reply:* I think I’m mistaken. Sometimes it exits and sometimes it doesn’t. I haven’t yet figured out all the context, but I know that when the setTimeout rejects, the process does exit.
*Thread Reply:* Wrong again - it does finish, it just… takes a heck of a long time…?
*Thread Reply:* log is now available in 7.0.0
That and attach are really about providing data to formatters. Do you just want to output something to the console?
*Thread Reply:* Yes, to the console. People on our team, me included, find it really annoying to download files from github to see the report and we don’t have resources at the moment to set up some kind of reporting page.
*Thread Reply:* formatters are for building reports (or similar) with the output of a test run
*Thread Reply:* depending on how much and how big, you could just console.log
*Thread Reply:* Thanks. We wanted it to appear after the cucumber test report, but that part isn’t that important, so maybe we’ll stick with console.log. A formatter might be a stretch goal.
Cucumber.js 7.0.0 is out!!! • https://github.com/cucumber/cucumber-js/blob/master/CHANGELOG.md#700-2020-12-21 • https://www.npmjs.com/package/@cucumber/cucumber
*Thread Reply:* Not something I’d expect, no - would you mind creating an issue under https://github.com/cucumber/cucumber-js/issues with some more details (ideally a little project where we can reproduce it) and I’ll take a look
*Thread Reply:* Sorry for the late reply @Jim Trenowden
I think the docs are at odds with the reality here. If you pass a buffer or stream as the data (first arg) it will be base64 encoded, but if you pass a string it will be assumed to be the content type you declare (i.e. not base64 encoded). That leaves us without a way to attach directly from webdriver at present since it's returning a base64 encoded string (as opposed to say puppeteer which will give you a buffer).
I'm going to raise on GitHub and try to get it addressed. In the meantime I think you could (kind of awkwardly) work around it by decoding the base64 string (I know), like:
await driver.takeScreenshot().then(function (screenShot) {
world.attach(new Buffer(screenshot, 'base64'), 'image/png')
})
*Thread Reply:* Sorry, just seen your reply. That’s very helpful, thanks. Good to know it’s not just me!
*Thread Reply:* Are you using a custom formatter?
Are you able to link to a minimal repro somewhere public?
*Thread Reply:* it is not mandatory per se from a runtime perspective, but it is necessary to make the compiler happy if you access anything on this in your step definition (and you use typescript).
*Thread Reply:* aham.. I see. I forgot to mention that Im using Typescript, yes. But in the the previous version it wasn't necessary...
*Thread Reply:* what do you mean by "it was not necessary"? do you get an error that was not there in previous versions, when you don't define it?
*Thread Reply:* With the previous version if I didnt pass the this: World as a parameter it worked fine. I could use this.foo within the step definition without any problem.
But now, it says the following:
error TS2683: 'this' implicitly has type 'any' because it does not have a type annotation.
and
When(/^the {string} foo bar tap$/, async function (type: string) {
~~~~~
An outer value of 'this' is shadowed by this container.
*Thread Reply:* I gonna check if adding the parameter this: World it solves the issue
*Thread Reply:* I mean the metrics from any test run (number of scenarios failed, passed, duration etc..)
*Thread Reply:* ah oki sorry. Good question I'm not 100% sure. I do know this kind of thing should be in the messages so if you're using the new-er versions it should be obtainable somehow.
*Thread Reply:* yeah we aggregate this from the standard messages emitted when cucumber runs
*Thread Reply:* if you look at the code for the summary formatter, you should be able to follow how it's doing it, and you could do something similar in a custom formatter
https://github.com/cucumber/cucumber-js/blob/master/src/formatter/summary_formatter.ts
*Thread Reply:* I also use Chai, but I'm testing webpages and I find that often it's useful to just keep retrying the action until the step times out. For example, if I expect to see a value in an element as the result of an ajax call it's more stable to repeatedly look at the element at 50 millisecond intervals to account for load lag than do a flat assertion.
*Thread Reply:* Hmm, yes assertions can lead to some unwanted early fails. I learned about them a bit later in my automation journey and then wanted to use them everywhere, but now I know that in fact navigating to a page and finding an element actually counts as an (implicit) assertion (if you found what you were looking for, you're on the right page).
*Thread Reply:* At the end of the day, Cucumber is looking for a thrown and uncaught error and will stop the test when that happens. That's all that's really needed. It is often useful to catch Puppeteer/Selenium errors, which often have cryptic error message text, and rethrow them with a message that will be clear to whoever is reading Cucumber's test output.
*Thread Reply:* Assertion libraries are one way to do this very quickly, but not the only way.
*Thread Reply:* @Michael Morris Oh, yes. I also have experienced inconvenience in adding timeout to assertion with assertion libraries. Thank you! I need to consider it again.
*Thread Reply:* Can you post more detail about your issue? Maybe put the code up on GitHub?
*Thread Reply:* @David Goss cypress-cucumber-prepocessor generates json report file and also screenshot of errors and on demand in a separate folder. You can the use Jenkins Cucumber plugin to see report with CSS added. There will be no screenshots attached to it unless you embed those screenshots into those json file(s)
*Thread Reply:* well, I do use both from above. But I didn’t find a reliable solution to embed on-demand screenshots into json files
*Thread Reply:* @Jo Laing what’s the process these days for community blog contributions? Can you help Michael out?
*Thread Reply:* @Michael Morris I don't know if we do that sort of thing anymore, but if you PM me what it is, maybe it's something I can include in the community newsletter.
*Thread Reply:* @Michael Morris Do you have the content ready in a doc? If so, you can post it in the <#C5YEEHG1E|recommended-media> channel for now and as Tracey mentioned above, it's maybe something we can include in the community newsletter too? I'd like to be able to offer up a place on cucumber.io for community blog contributions and have started looking at how to make that happen. Hope that helps a bit for now! 🙂
I personally think this one is the best one 🙂 https://cucumber.io/blog/open-source/tweaking-cucumber-rubys-auto-loader/
*Thread Reply:* yes. I am running different jobs on ci and then I collect all outputs and would like to generate a single report. Works well with cucumber6
*Thread Reply:* Good question - we don't seem to have any coverage or documentation of the interplay between --retry and --fail-fast but I would say yes, I think if you have both it should exhaust the retry attempts for a scenario and then fail fast (as opposed to disregarding the retry).
*Thread Reply:* I wrote this method for this problem.
applyTraits: (klass, ...traitLibraries) => {
for ( const traitLibrary of traitLibraries ) {
const traits = require(paths.modules + "/traits/" + traitLibrary);
for (const [trait, implementation] of Object.entries(traits)) {
if (klass.prototype[trait]) {
throw new Error('FATAL: Trait method ' + trait + ' already defined in the class.');
}
klass.prototype[trait] = implementation;
}
}
return klass;
}
*Thread Reply:* It's a limited implementation of the Ruby notion of traits using the JavaScript prototypical inheritance.
*Thread Reply:* You would pass your World class as the first argument, and the rest of the function can be overloaded with library names. Here's a call to the function from my code: const Page = loader.applyTraits(loader.getClass("Page"), "Actions", "FormMethods", "WaitMethods", "UcxFields");
*Thread Reply:* Each trait library is simply an object of callbacks.
*Thread Reply:* Hmm.. Reading closer you're moving in the same direction as what I posted. What you have should work if the body of foo.js is this:
module.exports = function() {
//do stuff
}
HOWEVER!!!
This will NOT work
module.exports = () => {
// do stufff
}
Arrow functions do not have a bindable scope, and bind does nothing to them. You can attach one to the world object, but it won't have access to this
*Thread Reply:* The method I posted above is looking for the trait libs to look like this:
module.exports = {
foo: async function () {},
moo: function () {},
bar: function (arg1, arg2) {}
}
And so on.
*Thread Reply:* This workaround works, but it's pesky
defineStep(/I log in as an? (administrator|manager|read-only user)/, async function(permission) {
await login.bind(this)(permission);
});
*Thread Reply:* ? after a capture or group means 0 or 1 of the preceding one. The same way as an? means the same as a(?:n)?
*Thread Reply:* What version of cucumber-js itself are you running locally?
*Thread Reply:* Okay - Rule is supported from 7.0.0 onlyhttps://www.npmjs.com/package/@cucumber/cucumber
*Thread Reply:* Right, thanks @David Goss looks like its still prerelease too: > 7.x.x and above > (Not yet released)
*Thread Reply:* Don't worry, there is a stable 7.0.0 out! Maybe a small hiccup in the release process there with that readme.
*Thread Reply:* I'll upgrade and see how I go, I think i'll need a new html reporter too
*Thread Reply:* @Des Horsley cucumber-js 7 has a new HTML formatter built-in
*Thread Reply:* @Chats I'm not sure I understand what you're asking for.
*Thread Reply:* Do you want to load test data from an external file?
*Thread Reply:* @Aslak Hellesøy Yes my external file (.json) in some folder and i wanna pass the file name as a string in examples ( jsut for readable purpose ) and i wanna write a logic where i can read the file by matching the string passed in examples of scenario and get the values of json
*Thread Reply:* @Aslak Hellesøy I have a class written in Java which is a UserProfileReader class but i am struggling to implement it in Js
*Thread Reply:* Hi @MarkSweetlove - I'm afraid there isn't. What do you need it for?
*Thread Reply:* In ruby it allowed us to add a custom formatter at runtime to avoid specifying it on the command line every time.
*Thread Reply:* @MarkSweetlove you can define common command-line options in a cucumber.js config file in your project root: https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#profiles
*Thread Reply:* Here is cucumber-js' own config: https://github.com/cucumber/cucumber-js/blob/master/cucumber.js
*Thread Reply:* On Ruby we could update the cucumber config from our own third party gem by registering the hooks which was ideal for our scenario, but doesn’t sound like that’s an approach we can take in JS.
@Huck Carignan have you followed the upgrade instructions? https://github.com/cucumber/cucumber-js/blob/master/docs/migration.md
*Thread Reply:* I got it running, but I'm now having issues with the @tags. I believe it might be an issue with the formatter changes. I won't be able to investigate this for another few hours (other work priorities)
*Thread Reply:* @Chats Just read the file in your step definition
*Thread Reply:* @Aslak Hellesøy i want to have it dynamic , as json passed in examples of scenario will be the name ( |some file1| and few more files as well |some file 2| and these files are under some UserProfile folder ( somefile1.json,------> now i wanna write a code in js which which will pick that json file from UserProfile folder by replacing some file1 to somefile1.json and read the file and return the values which eventually i want to use it in functions.
*Thread Reply:* @Chats Can you write your step definition so it receives the base name of the JSON file as an argument?
*Thread Reply:* When you have done that, just use fs.readFile to read the contents of the JSON file.
*Thread Reply:* @Aslak Hellesøy, Thanks , I will try to implement this solution
*Thread Reply:* So the error code is very declarative. It's telling you your fetch is failing on the index. So before you do anything else. Debug the line before execution and return the 1d-array in both versions. Triage from there.
*Thread Reply:* Looks like index is out of bounds, or the array has undefined values (less likely). Something might be wrong in coordinator.ts. BTW we need source maps to get better stack traces. Any idea @charlierudolph @David Goss ?
*Thread Reply:* Here's the line of code https://github.com/cucumber/cucumber-js/blob/master/src/runtime/parallel/coordinator.ts#L108
*Thread Reply:* Seems to suggest the coordinator and worker don't have the same information about the step definitions somehow.
*Thread Reply:* @Jim Trenowden would you be able to share something that would help us reproduce this?
*Thread Reply:* I'm not super familiar with this area so first step is to reproduce. @Jim Trenowden would it be practical for you to post your project to github (or an edited versin of it)?
*Thread Reply:* @Aslak Hellesøy we do ship source maps in the package, but not the actual TypeScript source files, which explains that. I'll do a PR for that soon.
*Thread Reply:* Thanks for looking into this! It’s a huge project at my workplace so not really practical to share it (not my IP strictly speaking, even though I wrote it), but the file calling the Cucumber CLI looks like this if that’s helpful?
```const cucumber = require('@cucumber/cucumber')
async function cucumberRunner(tags, scenarioPath, parallel, jsonPath) {
const argv = process.argv
const cucumberArgs = argv.concat(
'--parallel',
parallel,
'--tags',
tags,
'--format',
json:${jsonPath},
scenarioPath
)
return await new cucumber.Cli({ argv: cucumberArgs, cwd: process.cwd(), stdout: process.stdout, }).run() }
module.exports = { cucumberRunner }```
*Thread Reply:* Thanks @Jim Trenowden I must admit I'm not at all familiar with the parallel implementation. But I know enough about parallelism that it's difficult to implement without concurrency bugs!
*Thread Reply:* i’ve noticed that if I knock the parallel threads down to 1 the first run of tests still passes fine, but in the second one the steps are undefined
It’s worth noting that the app that calls the above file is a Node.JS app that stays alive inbetween test runs if that makes a difference
*Thread Reply:* Hmm. Perhaps we have some sticky state somewhere that prevents Cucumber from running properly more than once in the same Node process?
*Thread Reply:* Possibly related: https://github.com/cucumber/cucumber-js/issues/1278#issuecomment-580603152
*Thread Reply:* Source maps fix is here https://github.com/cucumber/cucumber-js/pull/1607
*Thread Reply:* Ah interesting, yeah issue 1278 does sound similar to what I’m seeing. Glad it’s not just me then!
*Thread Reply:* Although it’s weird that this only started happening in v7, in v6 this all works absolutely fine
*Thread Reply:* So clearing the Cucumber module cache between runs seems to “resolve” this, seems a bit hacky but it’s unblocked me at least. Thanks!
*Thread Reply:* The exact feature sets to be tested depend on the brand theme in use, and the step definitions change as well based on this.
*Thread Reply:* npm adds ./node_modules/.bin/ to the PATH so no need to add that.
You can also execute arbitrary commands from npm scripts so just extract to a shell script
*Thread Reply:* That's not what I'm asking. Look at the god-awful argument list being passed to cucumber - that's what I'm asking about please - how to get rid of that.
*Thread Reply:* I also recommend adding common cucumber options in cucumber.js - see https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#profiles
*Thread Reply:* Here is cucumber-js' own config: https://github.com/cucumber/cucumber-js/blob/master/cucumber.js
*Thread Reply:* But I still need to start a hugo webserver and an express.js server, preferably before Cucumber does anything.
*Thread Reply:* No, my preference is really to do it in a Before hook. That way I have less of a chance having server state accidentally leaking between scenarios
*Thread Reply:* That would add 2 minutes to all scenarios and cause the tests to run for 4 hours. So, no.
*Thread Reply:* Besides, in my case that's an unwarranted concern. I'm testing static javascript files built by Hugo, not a webserver that carries any sort of state.
*Thread Reply:* Yeah, if you have no state I agree BeforeAll is better.
*Thread Reply:* I do use a Before hook to force a reset of the Express server though.
*Thread Reply:* But that thing comes up in microseconds as opposed to a full minute or two.
*Thread Reply:* Yeah, this will help a lot. I'm reading the profile information now.
*Thread Reply:* Even though it wasn't the question I asked, I didn't realize ./node_modules/.bin was on path so that does help.
*Thread Reply:* Thank you for your time and I'm sorry if I came across as snippy - I've been dealing with a really weird and frustrating bug this morning.
*Thread Reply:* I have a core application, and currently three brands. The brands have their own features, and the core specifications have different step definitions. I could use tagging as you describe and keep the features in one folder, but I'd have to write skip logic to skip tests belonging to brands other than the current one.
*Thread Reply:* I've built a test runner that starts cucumber off as a child process. It also goes ahead and starts the Hugo Web Server and Express.js Mock server, and stops them when the Cucumber process fires its exit event. The approach will work, though I wish I had an easy way to relay command line flags to Cucumber.
*Thread Reply:* @Michael Morris I dunno if this will help in your context, but I worked on a set of feature files the BBC had to specify a TV app that was run on lots of different TV’s “browsers” and a Flash-based set-top-box. We used the same set of feature files to test each TV, but tagged out scenarios that were not supported / applicable on some platforms. e.g. @not-samsung-tn482
You might find it’s more manageable to have all the specifications in one place and just tag out the discrepancies.
*Thread Reply:* I'm facing something very similar actually. One of the reasons that I'm wanting to keep the features in two directories is we are considering putting the brand themes off into their own repository modules. That would require their features stay with the rest of the code.
*Thread Reply:* I was tagging them earlier though and I didn't remove the code supporting it - @theme=ifleetfor example.
*Thread Reply:* Tags should be working as before. Can you elaborate on what you’re seeing?
*Thread Reply:* yea, I was just about to edit my above question so that is clearer. 🙂
*Thread Reply:* That was working fine on the original cucumber-js module
*Thread Reply:* Weird question, but are you using any particular formatters? I've noticed issues with tags and the progress-bar formatter in v7
*Thread Reply:* So I've had to stop using the progress-bar formatter as I've noticed that it will sometimes ignore the tags provided and run all scenarios in the feature path
*Thread Reply:* Whereas not using that formatter seems to make everything behave as expected in v7
*Thread Reply:* Haven't tested other formatters to see if they have the same issue
*Thread Reply:* sounds like we're referring to https://github.com/cucumber/cucumber-js/issues/1579
*Thread Reply:* which is a bug where the progress bar formatter incorrectly reports the total number of steps (not accounting for the tag filter)
*Thread Reply:* but the formatter can't influence what's run or not run
*Thread Reply:* Ah-ha! Yes, knew I wouldn't be the first to spot this. Good to know it's been raised
*Thread Reply:* so i think it's likely that your tag expression is working (only the scenarios you wanted are run) but it just doesn't look that way from the output
*Thread Reply:* Run it with or without - the progress bar is just misreporting
*Thread Reply:* I'm trying now without the progress bar to see what happens
*Thread Reply:* Yup, I believe you gentlemen are correct. Just a bug with the progress bar - not the test cases being run
*Thread Reply:* Specifically, I need both of them to see the browser configuration that I pass in as world params. When I have the headless mode of puppeteer turned off I want to leave the browser running when the test concludes. This allows the developer to interact with it after the test to work on things.
*Thread Reply:* My current work around is to put Before, BeforeAll and AfterAll in the same file. The browserConfig is a variable all these closures can see. Before inits the browser once only since BeforeAll can't without the config. AfterAll then looks at that var to determine if it's turning off the browser.
*Thread Reply:* This may happen soon-ish, see https://github.com/cucumber/cucumber-js/issues/1393
But also, would generally recommend an instance of puppeteer per test (world) rather than sharing one across the whole test run. Ensures segregation between tests and allows for running in parallel.
*Thread Reply:* As mentioned, there are things that would be nice to share in the world in beforeall /afterall. But each beforeall / afterall currently spins up a new world instance so it's a bit intractable>
Also as David said, definitely don't share browser state. Should tear up / tear down the browser
For reference anyway, there's a migration guide here https://github.com/cucumber/cucumber-js/blob/master/docs/migration.md - feedback welcome
*Thread Reply:* I’m afraid I’m not familiar enough with the code these days to know. @David Goss should know.
*Thread Reply:* Hey @MarkSweetlove, we do have a bit of a gap here. There is a testCase message which is effectively what you want - only emitted for a compiled scenario that is going to actually run - but at the moment the testCase message is emitted just before that test case executes. To have an accurate progress indicator we need to emit them all upfront before any tests execute - I'm actually working on this now under https://github.com/cucumber/cucumber-js/issues/1408, because I need it to solve https://github.com/cucumber/cucumber-js/issues/1579 at which point the progress bar formatter should serve as a good reference point for your use case. Please go ahead and watch those issues and hopefully this will be resolved pretty soon.
*Thread Reply:* Thanks David. Appreciate your response and I’ll watch those issues excitedly!
*Thread Reply:* We’re going to make world parameters available to BeforeAll and AfterAll hooks soon, see https://github.com/cucumber/cucumber-js/issues/1393
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* There's also https://github.com/darrinholst/eslint-plugin-cucumber for linting your step code, if that might be useful to you.
*Thread Reply:* It seems to be related to the following issue: https://github.com/cucumber/cucumber-js/issues/1646
*Thread Reply:* yep - think we'll need to revert and then publish a 7.2.1 so we can regroup - i'll do that today
*Thread Reply:* @Tally Barak 7.2.1 is out which should unblock you - sorry about that!
*Thread Reply:* This looks like a warning from eslint. Does the code execute as expected @Deb Learns?
*Thread Reply:* Good, not sure how to get rid of that warning though.
*Thread Reply:* Yeah the linter has no way of knowing that cucumber-js is going to bind the world to that function internally - a bit of a non-deterministic rule on that basis really. You could consider disabling that rule for your step_definitions specifically.
*Thread Reply:* If you feel like using TypeScript for your step definitions, you can then tell the compiler what this will be, see https://www.typescriptlang.org/docs/handbook/functions.html#this-parameters
*Thread Reply:* Hello 🙂
Here's the .feature file for the rerun formatter:
https://github.com/cucumber/cucumber-js/blob/1109951440833c7f074b04d1b294e96ff1be65eb/features/rerun_formatter.feature
The idea is to use that formatter to output failures in a specific file which will then be used as an argument to a second execution of cucumber.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Thanks @Aurélien Reeves! I was thinking another way to retry automatically multiple times without reporting. But maybe report it and run it failed one again is better approach. 🤔
*Thread Reply:* @Tim Yao https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#retry-failing-tests
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* @Aslak Hellesøy That’s what exactly I am looking for! Thanks! Will try it Monday.
*Thread Reply:* @plocket there is no way to do this currently. I suggest you use column values with a custom toString() method. Or, you could open an issue where we could discuss adding a feature that lets you control table formatting.
*Thread Reply:* I’m not sure how/where I’d use a custom toString() method on column values, but I do know how to make an issue. Thank you.
*Thread Reply:* I forgot that in JavaScript you cannot control the type of table cells (you can in Java). See you in the ticket.
*Thread Reply:* Also strange, is that the same basic logic works perfectly fine when the scenario ends:
if (envelope.testCaseFinished != null) {
if (envelope.testCaseFinished.testCaseStartedId == null) { return }
const testCaseAttempt = this.eventDataCollector.getTestCaseAttempt(envelope.testCaseFinished.testCaseStartedId)
const parsed = formatterHelpers.parseTestCaseAttempt({
cwd: this.cwd,
snippetBuilder: this.snippetBuilder,
supportCodeLibrary: this.supportCodeLibrary,
testCaseAttempt,
})
<a href="http://console.info">console.info</a>(parsed)
}
this is not throwing any errors
*Thread Reply:* I don't know the code, but I've had a look. It seems like the error you're getting is probably thrown from here:
https://github.com/cucumber/cucumber-js/blob/v7.2.1/src/formatter/helpers/test_case_attempt_parser.ts#L116 (I guess the line numbers are different in the typescript source?)
If that's true, this suggests the testStepResult is undefined in that function.
That function's called here: https://github.com/cucumber/cucumber-js/blob/v7.2.1/src/formatter/helpers/test_case_attempt_parser.ts#L169
and the line above it that sets the testStepResult is here:
https://github.com/cucumber/cucumber-js/blob/v7.2.1/src/formatter/helpers/test_case_attempt_parser.ts#L157
So it seems like the code is assuming that every one of the testSteps will have a corresponding entry in the stepResults map, and that this isn't the case here?
It seems like this is a bug; I wonder if you could add some debugging into the cucumber source code on your machine to see if we can pinpoint what's wrong?
*Thread Reply:* Anything you attach from a step will come up in the report (long as it's in some displayable form). Are you able to say more about what you want to show and how?
*Thread Reply:* Sure, so previously I was using the cucumber-html-reporter package for our reports, and that allowed us to attach custom metadata to the “top” of the report that related to the whole test run, rather than a specific step or scenario.
Some of the metadata we were adding is at the top of the new HTML reports already, such as duration, OS, Node JS version etc. But we were also attaching things such as the feature path and tag combination that was used to kick off the test run, test environment, browser (as we’re using Cucumber to kick off Selenium tests) etc.
The data at the top of the new HTML reports seems to be coming from the meta object in the message formatter, if there was an easy way to inject custom metadata in there that would be great
*Thread Reply:* Can you give an example (in JSON) of what you'd like to attach, and how you'd like it to be displayed?
*Thread Reply:* Hello @Kit Peters 🙂
Here's a simple example of an app and its cucumber tests: https://github.com/cucumber-ltd/shouty.js
As you can see in https://github.com/cucumber-ltd/shouty.js/blob/master/features/step_definitions/shout_steps.js, it requires the code from your app, and uses the function as you would do with any module
Will that help?
<a href="https://github.com/cucumber-ltd/shouty.js">cucumber-ltd/shouty.js</a>
*Thread Reply:* @Aurélien Reeves I'm sorry, I wasn't clear. The message at the root of this thread follows on from the message before it. I don't need to know how to write a standard test using Cucumber; there are numerous good examples (such as yours) of that. What I need to know is how, at a functional level, to process a user-defined scenario using Cucumber and get the result of that processing in a variable that I can then write to a database.
*Thread Reply:* I'm not sure I understand. Let me state it another way. I want a function that takes a scenario in the form of an array of strings and returns a boolean indicating whether or not the scenario was successful.
*Thread Reply:* I'm trying to use Cucumber in an application context, not a test context.
*Thread Reply:* Why would you need the scenario in the form of an array of strings ?
Beside that, I would say that what you are looking for is a custom formatter. You can take a look here to take a look on how to build your own formatter : https://github.com/cucumber/cucumber-js/blob/main/docs/custom_formatters.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I need the scenario in the form of an array of strings because the scenario will be supplied by the user. I will not be defining the scenario beforehand.
*Thread Reply:* And the custom formatter doesn't help me get my scenarios into Cucumber in the first place.
*Thread Reply:* I have an idea that might help explain what I'm trying to do.
*Thread Reply:* Suppose I were to write a command line application that would allow the user to input their own scenarios via the keyboard. How would I send the user input to Cucumber?
*Thread Reply:* Ok, I understand now 😛
Right now I do not have the time to go into much details. I will look deeper tomorrow.
I think what need to be done is: • generate a feature from those string arrays • parse it with gherkin: https://github.com/cucumber/common/tree/master/gherkin/javascript • run the resulting pickels with the cucumber-js runner: https://github.com/cucumber/cucumber-js/tree/main/src/runtime • then use a dedicated formatter for the post process of the results It is actually not trivial, I am not sure we have an API that would make that process easier, but I still do not know that much the API 😛
Maybe @David Goss or @charlierudolph would have better ideas?
*Thread Reply:* I've been speaking with @David Goss in another thread, but he hasn't responded to my last message. I haven't spoken with @charlierudolph.
*Thread Reply:* Those steps look very close to what I'm trying to accomplish.
*Thread Reply:* To clarify, a bug in the cucumber js logic in writing out the messages to cucumber.ndjson?
*Thread Reply:* Yes, as I understand it we have duplicated formatters in each of the cucumbers that write out this JSON format.
It's a long weekend in Europe, so might be best to raise an issue on cucumber-js so this doesn't fall through the net.
*Thread Reply:* I think this is https://github.com/cucumber/cucumber-js/issues/1534
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Last change to fix this will land next week (depends on https://github.com/cucumber/cucumber-js/pull/1645 but nearly there)
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Yes, that sounds like the issue, thank you. I will check back in after that lands.
*Thread Reply:* @David Goss, when do you think this fix will be released?
*Thread Reply:* That plugin is not supported by us. I suggest you ask the authors of the plugin @SantoshK
*Thread Reply:* If a hook fails to do what it needs to do, it’s best to throw an error as you have suggested.
*Thread Reply:* @Michael Stelly this one is pretty good: https://turriate.com/articles/modern-javascript-everything-you-missed-over-10-years
*Thread Reply:* @Michael Stelly the change log is probably the best place to look.
https://github.com/cucumber/cucumber-js/blob/main/CHANGELOG.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Let me rephrase: I'm looking for a current guided walkthrough to help me understand the workflow for cucumber similar to this 3-year old blog post http://www.thinkcode.se/blog/2018/02/07/getting-started-with-cucumber-for-javascript.
*Thread Reply:* I found the example file in the repo. That should help to get me started.
*Thread Reply:* Yes, it's possible.
Cucumber scenarios (whether they come from regular scenarios or examples in a scenario outline table) are intended to be independent, to avoid the
Given this, here are your options:
- Ignore our advice, and add a "Log in" scenario at the top of your feature file. Always run the scenarios in order so this one runs first.
- Add a tagged hook that ensures your session is logged in. Add the tag to each of your scenarios that need the user to be logged in. Make the code in the hook clever enough to check first whether the user is logged in, and only do the login if necessary.
- Do something similar to (2) in an explicit Given step instead of a hook.
*Thread Reply:* So we have this situation. And I would advocate a fourth option - But it's not test architecture, it's software architecture. (Matt also would probably push for this irl).
If the logging in isn't something you care about testing, and it's something which is causing a pain point in terms of test execution and it's either not directly your system, or a system you are confident in it's existing functionality, then adding a bypass or backend is advised.
*Thread Reply:* at CA we have to login using a different website currently then redirect and pass a cookie, so we just artificially generate this. It removes 99% of the logging in time.
You just create it in your project's root directory. Documented here: https://github.com/cucumber/cucumber-js/blob/main/docs/cli.md#profiles
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* ahh Thank you. I didn't know it was a custom config file. I assumed it was something already created.
*Thread Reply:* I appreciate the help during my learning curve. I'll be sure to pay it forward.
*Thread Reply:* Great, glad you're liking them. Please post any feedback directly to the team in <#CCK3ZFSF9|school-devs> - we're still working on the last few episodes.
Has anyone used the new(ish) @cucumber/gherkin-streams package? I've been having some trouble requiring it: https://github.com/cucumber/cucumber-js/issues/1675
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Crosspost:
If you try something like the following, does it work? ```const { GherkinStreams } = require('@cucumber/gherkin-streams')
const stream = GherkinStreams.fromPaths(/.../)``` If so, it seems to be an error in the documentation
Does anyone have any insight into this comment (https://github.com/cucumber/cucumber-js/issues/1489#issuecomment-820716265) around getting test cases from the file system? Also do you have an update on this 6 month old issue (@Aslak Hellesøy) I still haven't been able to resolve? If so a response on the issue would be much appreciated 🙂
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Issue updated @Aurélien Reeves and thanks muchly for your eye balls and grey matter on it!
*Thread Reply:* 👍
As soon as I have a couple hours to take a look on it, I'll do
*Thread Reply:* I'd expect with the "minimal reproducible example" you requested it should be minutes rather than hours @Aurélien Reeves? Thanks.
What kind of detail do you want to output? The pretty formatter might be what you want https://www.npmjs.com/package/@cucumber/pretty-formatter
*Thread Reply:* Thanks for answering. It's not a formatting issue. I'd like to see output like this
*Thread Reply:* Apparently, that's not a cucumber question. Probably a reporting feature in mocha.
*Thread Reply:* But I wasn't aware of the Prettier for Cucumber. So that's a good reference for later.
*Thread Reply:* To be more clear, I get output for passed steps IF I have a failing step. But if all pass, I simply get pass notification.
*Thread Reply:* Ahh. Now I see. The lesson on configuring cucumber with this formatter doesn't appear until later. I just arrived at it.
*Thread Reply:* Cool. This is exactly what I was looking for: ``` Rule: Shouts can be heard by other users
Scenario: Listener hears a message # features/hear_shout.feature:7
Given a person named Sean
And a person named Lucy
When Sean shouts "free bagels at Sean's"
Then Lucy should hear Sean's message```
Ping on release of fix for https://github.com/cucumber/cucumber-js/issues/1534 , looks like it has been addressed and merged into master but has not been included in a release yet.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* What are the considerations for a release? Otherwise my question was asked due to ignorance.
*Thread Reply:* Typically when there is a consensus amongst the core team that we're ready for a release. We meet online on Thursdays to discuss. Then, if we feel a release is ripe, somebody has to spend time doing a bit of manual testing, write up release notes and make a release.
*Thread Reply:* So my current iteration looks like this ```Background: Consumer is shopping Given the consumer is shopping on the app
Scenario: Favorite an item When the consumer finds a desired item Then the app should allow the consumer to add it to the favorites list``` No buttons, no checkboxes for size/quantity. Just adding something to a list.
*Thread Reply:* The Given step is not using past tense though, which might be a signal something's wrong.
*Thread Reply:* Remember you can specify Rule: now in Gherkin too. Cleaving between rules and examples is a really important part of this process.
*Thread Reply:* The entire feature looks like this: ```Feature: Add item to Favorites
As a consumer (guest or member) who is shopping on the App, I'd like the ability to favorite items that I maybe interested in.
Rule: Any consumer, guest or member, can favorite an item after selecting a size, quantity, and shipping method
Background: Consumer is shopping Given the consumer is shopping on the app
Scenario: Favorite an item When the consumer finds a desired item Then the app should allow the consumer to add it to the favorites list``` Let me think on the "given is past tense" comment
*Thread Reply:* I wonder if you need more stuff like a given to set up the state that there is a "desired item" in your database somewhere for them to find.
*Thread Reply:* I don't think that's implicit in "the consumer is shopping on the app"
*Thread Reply:* As I mentioned, this is a PoC to introduce BDD to a naive audience - the product team.
*Thread Reply:* I like your scenario title. Follows the "Friends episode" heuristic nicely.
*Thread Reply:* I want to keep it light on details so that I don't overwhelm them. In terms of our crawl/walk/run stage, we're not even on our hands n' knees, yet.
*Thread Reply:* Would a better "given" be Consumer opened the app? That seems trite and empty. I'm a bit confused on where to draw the line between the declarative behavior and the imperative implementation.
*Thread Reply:* Have you added any automation under the hood yet?
*Thread Reply:* No, this is the very start of our foray into BDD. I'm still trying to get a grasp of the concepts. Here's my revised background.
Background: Consumer searched the product inventory
Given the consumer searched the product inventory
I'm viewing the Cucumber School videos and trying to apply the concepts to this selected feature as a real-world example.
*Thread Reply:* The magic of Gherkin is that it sits on the problem/solution domain boundary. You're working hard here to make it work in the problem domain, but I think you've started to spin the wheels a bit as you haven't had any feedback from the solution domain yet.
I'd suggest at least thinking about how you'd automate each of these steps, and then consider whether you've missed any additional steps that would be necessary to actually automate the whole scenario (assuming empty state in the system at the beginning), and then how you would express those in a way that makes sense in the problem domain.
*Thread Reply:* I have a rudimentary start to that, like the beginnings of a Consumer and Network class. This is very much the "fix (enhance) the car while driving it" situation. The app in question is a live production mobile app. All the infrastructure and supporting data are already in place. Add to Favorites is a completely new feature which is why I selected it for the demo.
As the only proponent of BDD in the organization, I have to rely completely on outside help like this forum, videos, tutorials and whatever else I can glean from the internet, to guide me. Your help is invaluable.
*Thread Reply:* Of course, now this begs the question: what are the feature boundaries? How difficult is it to start using BDD for an already mature, production app?
*Thread Reply:* I found the SmartBear youtube channel. This webinar outlined all the points you were helping me understand. Hope it can help others. https://youtu.be/awwFfCYoGFQ
*Thread Reply:* Ha nice one! @Jo Laing I guess we need to do a better job of signposting people to these videos. (Or we could re-publish them on school?)
*Thread Reply:* While we're talking youtube there's also a Cucumber channel with a lot of interesting talks from our conference:
https://www.youtube.com/channel/UC0eV4MhnmCd-48XdDhzf6Hg
Hi, Is there anyone who has experience with the WebdriverIO custom reporter? Or with CucumberJS reporter? I would like to integrate the Cucumber test results into the Testrail, but I am stucked a little. I installed babel when I initialised the wdio project and I followed this instruction: https://webdriver.io/docs/customreporter but I am confused now: ```import WDIOReporter from '@wdio/reporter' ^^^^^^
SyntaxError: Cannot use import statement outside a module```
*Thread Reply:* I don't think this is an issue with either of the reporters.
Before you use imports you need to make sure that, at the starting point of your code execution, you have Babel. Since wdio launches from the conf file, that is where you want to put it. So you will need a require here, but no-where else.
In wdio.conf.js include the following:
// this allows wdio.conf.main.js to use es6 imports
require("@babel/register")({
presets: ["@babel/env"],
});
module.exports = require("./wdio.conf.main");
Put what you would normally have in wdio,conf.js in wdio.conf.main.js (or whatever name you find appropriate, just match it to the require in the file above.
You should be able to use es6 imports everywhere in your code after that.
*Thread Reply:* I uploaded my project here: https://github.com/peterzsilak/wdio-cucumber-testrail/tree/add_cucumber_reporter I can use imports in other files… Can you please check my code?
*Thread Reply:* Well, I just cloned it (I'm actually curious about the Testrail integration, it's something I have thought about but never looked into). But I should be working right now. Can you first try implementing my suggestion? It is a tip I got straight from the WebdriverIO community after much wrangling with this issue, I am confident it will resolve your issue.
*Thread Reply:* Sure, I did, but and now I have a new error:
Error: Type check for option "reporters" failed: a reporter should be either a string in the format "wdio-<reportername>-reporter" or a function/class.
*Thread Reply:* I committed the changes, if you want to check it. 🙂
*Thread Reply:* Do you have something like
// Test reporter for stdout.
// The only one supported by default is 'dot'
// see also: <https://webdriver.io/docs/dot-reporter>
reporters: ["spec"],
in your main wdio conf file?
Perhaps your reporter needs to go there.
*Thread Reply:* I suggest the WebdriverIO gitter community if you're still stuck
*Thread Reply:* reporters: ['spec',
['CucumberTestrailReporter', {}]
],
*Thread Reply:* [0-0] 2021-06-09T14:10:40.165Z INFO @wdio/local-runner: Run worker command: run
[0-0] 2021-06-09T14:10:40.361Z DEBUG @wdio/config:utils: Found 'ts-node' package, auto-compiling TypeScript files
[0-0] 2021-06-09T14:10:40.682Z DEBUG @wdio/local-runner:utils: init remote session
[0-0] 2021-06-09T14:10:40.684Z ERROR @wdio/runner: Error: Type check for option "reporters" failed: a reporter should be either a string in the format "wdio-<reportername>-reporter" or a function/class. Please see the docs for more information on custom reporters (<https://webdriver.io/docs/customreporter>)
[0-0] at Object.validateConfig (/Users/pzsilak/Projects/ReachAndAttract/wdio_cucumber/node_modules/@wdio/config/build/utils.js:68:27)
[0-0] at Object.remote (/Users/pzsilak/Projects/ReachAndAttract/wdio_cucumber/node_modules/webdriverio/build/index.js:48:29)
[0-0] at Object.initialiseInstance (/Users/pzsilak/Projects/ReachAndAttract/wdio_cucumber/node_modules/@wdio/runner/build/utils.js:64:30)
[0-0] at Runner._startSession (/Users/pzsilak/Projects/ReachAndAttract/wdio_cucumber/node_modules/@wdio/runner/build/index.js:216:70)
[0-0] at Runner.run (/Users/pzsilak/Projects/ReachAndAttract/wdio_cucumber/node_modules/@wdio/runner/build/index.js:66:34)
[0-0] at process.<anonymous> (/Users/pzsilak/Projects/ReachAndAttract/wdio_cucumber/node_modules/@wdio/local-runner/build/run.js:24:30)
[0-0] at process.emit (events.js:327:22)
[0-0] at process.EventEmitter.emit (domain.js:467:12)
[0-0] at emit (internal/child_process.js:903:12)
[0-0] at processTicksAndRejections (internal/process/task_queues.js:81:21)
[0-0] 2021-06-09T14:10:40.685Z DEBUG @wdio/utils:initialiseServices: initialise service "chromedriver" as NPM package
[0-0] 2021-06-09T14:10:40.744Z ERROR @wdio/local-runner: Failed launching test session: Error: Couldn't find plugin "CucumberTestrailReporter" reporter, neither as wdio scoped package "@wdio/cucumbertestrailreporter-reporter" nor as community package "wdio-cucumbertestrailreporter-reporter". Please make sure you have it installed!
[0-0] at Object.initialisePlugin (/Users/pzsilak/Projects/ReachAndAttract/wdio_cucumber/node_modules/@wdio/utils/build/initialisePlugin.js:41:11)
[0-0] at BaseReporter.initReporter (/Users/pzsilak/Projects/ReachAndAttract/wdio_cucumber/node_modules/@wdio/runner/build/reporter.js:152:37)
[0-0] at Array.map (<anonymous>)
[0-0] at new BaseReporter (/Users/pzsilak/Projects/ReachAndAttract/wdio_cucumber/node_modules/@wdio/runner/build/reporter.js:22:50)
[0-0] at Runner.run (/Users/pzsilak/Projects/ReachAndAttract/wdio_cucumber/node_modules/@wdio/runner/build/index.js:77:26)
[0-0] at processTicksAndRejections (internal/process/task_queues.js:93:5)
2021-06-09T14:10:40.771Z DEBUG @wdio/local-runner: Runner 0-0 finished with exit code 1
*Thread Reply:* " Error: Couldn't find plugin "CucumberTestrailReporter" reporter, neither as wdio scoped package "@wdio/cucumbertestrailreporter-reporter" nor as community package "wdio-cucumbertestrailreporter-reporter". Please make sure you have it installed!"
*Thread Reply:* I see it also directs you here https://webdriver.io/docs/customreporter/
*Thread Reply:* So if that isn't enough to figure it out, again I would suggest the WebdriverIO Gitter community
*Thread Reply:* Look, I don’t understand this resolution…. when I installed the wdio I installed Babel also… And everything was okay, just this reporter did not wanted to work…
*Thread Reply:* I don't think I would be able to help further without really digging into it; it's not something I have worked on 🙂
*Thread Reply:* Okay, and thanks for your help,… I will look around in WebdriverIO Gitter. 😉
*Thread Reply:* • You could simplify the When section (yes). For example, you could obfuscate the size / quantity / shipping method details and move that logic to your helper steps, especially if the details are "incidental" to the test.
• You could use an example table.
*Thread Reply:* Ok. They're not incidental. They are pre-reqs. Before an item can be "favorited", all 3 -size/qty/ship values - must be chosen.
*Thread Reply:* Maybe I can say something like "when pre-reqs are selected" bc, who knows, maybe they could change at some point.
*Thread Reply:* Like this:
When I find a desired item
And pre-requisites are selected
*Thread Reply:* Sure, or even tweak the language slightly so it's just one step. "When I select a desired item with specified characteristics"
*Thread Reply:* You don't want that to get complicated though. The point being, what we're really testing is "adding the item to favourites" and we don't really care what the specific items are that we need to do before that, which, as you stated, could change (when that happens, you can change the implementation code, but would no longer need to change the scenario)
*Thread Reply:* Right. The scenario should indicate that there are conditions that must be in place before I can succeed.
*Thread Reply:* and that should be enough. What the conditions are, is not relevant here.
*Thread Reply:* The interesting thing about distilling steps like this is you have to find a name for the distilled concept. You've used "pre-requisites" here, for example. That then becomes part of your domain language.
*Thread Reply:* It's useful to check in with your domain-experts to make sure they like the terminology and buy into it.
*Thread Reply:* I think the Example table goes along with the Scenario Outline, doesn't it? 🙂
*Thread Reply:* I hadn't seen that multiple-cucumber-html-reporter package before. It's not something we maintain so unless another user or the maintainer are hanging out here you're going to be on your own 😞
There seems to be quite good docs at https://github.com/wswebcreation/multiple-cucumber-html-reporter#readme
What have you tried already?
*Thread Reply:* I'd like to export my cucumber tests in a report format (preferably HTML). That's my scenario. I don't have much inputs or commands for it yet and I'd like some inputs
Hi everyone - @cucumber/cucumber 7.3.0 is out! https://twitter.com/cucumberbdd/status/1405632173563170825
Twitter
*Thread Reply:* If i give npm install @cucumber/cucumber
How will I execute my feature file?
Is it ./node_modules/.bin/@cucumber/cucumber or how do I execute it?
*Thread Reply:* (Normally there shouldn't be any files called ./node_modules/.bin/@cucumber/cucumber)
You can also add this to your package.json:
{
"scripts": {
"test": "cucumber-js"
}
}
And run npm test (npm autimatically adds ./node_modules/.bin to your PATH)
*Thread Reply:* I use something similar
"scripts": {
"test": "cucumber-js -f @cucumber/pretty-formatter"
}
Then run yarn test from cli
*Thread Reply:* Use cucumber-js --format html:cucumber.html
Also see cucumber-js --help for full help about the command line options.
*Thread Reply:* (This should create a cucumber.html report in your current directory)
*Thread Reply:* There's also --publish which will create a HTML report automatically at https://reports.cucumber.io
*Thread Reply:* Hi @Mohamed Soliman are you also using `setWorldConstructor`` to use a custom class for your world?
*Thread Reply:* nope.. I just need the basic world to pass variables from CLI to stepdefinitions
*Thread Reply:* Are you using arrow functions or regular functions for your hooks/steps?
*Thread Reply:* In general, if you use arrow function, this will not have the value of your world object.
*Thread Reply:* There isn't anything we can do about this in Cucumber.js - it's a feature of the JavaScript language:
> Arrow functions establish "this" based on the scope the Arrow function is defined within.
*Thread Reply:* but it was not clear when reading through world.md document, I was trying for a whole day and it was not mentioned anywhere
*Thread Reply:* You should have asked sooner 🙂 - I'll add a note about this in the docs.
*Thread Reply:* Just noticed it's the first question in the FAQ: https://github.com/cucumber/cucumber-js/blob/main/docs/faq.md#the-world-instance-isnt-available-in-my-hooks-or-step-definitions
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* oh yes.. I missed it while searching.. Thanks, sir
*Thread Reply:* But it's easy to miss, so I'll add some more mentions of it 🙂
*Thread Reply:* @Mohamed Soliman hopefully this makes it clearer!
*Thread Reply:* https://github.com/cucumber/cucumber-js/pull/1705
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* For what it's worth it stumped me as well at first. I mention this in the commands of my tester's documentation for the benefit of my other team members.
*Thread Reply:* I've built a wrapping file to start my cucumber tests already. It can read the command line arg before the cucumber library even loads, but can I remove the arguments I've handled from the argv line? I've looked into cucumber deeply enough to know it's using commander as its cli reading dependency.
*Thread Reply:* Another tact would be to expand the capabilities of cucumber to allow the test writer to register additional command line flags.
*Thread Reply:* Also note - I'm well aware of profiles and they are very powerful. I use the world parameter to feed puppeteer its basic parameters already - mostly which device to emulate. BUT the showbrowser mode is something I would want to be able to turn on with any profile, so I can't really make it active in certain profiles. I also don't want to ask my users to edit the cucumber.js to turn the browser GUI on or off.
*Thread Reply:* Or maybe this is possible by working with Commander directly in the program flow...
*Thread Reply:* Ok, I have a solution. I can look for my custom flags before starting Cucumber, and if I see them slice them out of the argv array like so:
if (process.argv.indexOf('--show-browser') > -1) {
process.env.headlessBrowser = false;
process.argv.splice(process.argv.indexOf('--show-browser'), 1);
} else {
process.env.headlessBrowser = true;
}
*Thread Reply:* Later in the same file I then hand off process control to Cucumber like so:
require('@cucumber/cucumber/bin/cucumber-js');
*Thread Reply:* I really wish that hand off mechanism was documented somewhere.
*Thread Reply:* We don’t really have a programmatic API for running cucumber at the moment, this is something we’re going to be working on shortly.
I believe you can repeat the —profile arg and the profile contents are merged. World parameters are also merged together. So could have a profile that just adds your showBrowser option as a world parameter, and this could be activated in conjunction with any other profiles. So you could use that pattern to effectively build up custom CLI options by way of composable profiles.
*Thread Reply:* Ran a quick test to make sure I wasn't imagining this.
In cucumber.js
module.exports = {
"foo": `--world-parameters '{"foo":true}'`,
"bar": `--world-parameters '{"bar":true}'`,
}
And then
npm test -- -p foo -p bar
Yields
*Thread Reply:* or you could use an environment variable? That's what I usually do for this kind of thing.
Big thanks for working with us to get https://github.com/cucumber/cucumber-js/issues/1489 over the line @Aurélien Reeves !
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* There wouldn't happen to be a "definitionExists( pattern )" function in the api would there? If there is great, if not I fear I'll need to write my own define step function that keeps track of what is and isn't defined.
https://twitter.com/cucumberbdd/status/1410980312939745287
Twitter
*Thread Reply:* This is wonderful, thanks to everyone who put work into creating and publishing this!
I can't wait to give it a try.
I have two thoughts.
- One of the big benefits I've noticed from using screenplay is that interactions are composable, so you can start out with granular, imperative, building block interactions and use them to create higher-order interactions for domain-level actions that might involve multiple interactions with the system to achieve. Some versions of the Screenplay pattern call these "tasks". I don't see any mention of this from the readme. Is it currently possible with this framework?
- I've long thought that we could have some tighter integration between Cucumber and Screenplay, to be a bit opinionated about this being the "right way" to organise your automation code and make it super easy for people to use, for example by having the actor parameter type built in. I realise it's a bit early for that, but I wondered if you'd thought about that too, and about what it might look like?
*Thread Reply:* This might even inspire me to finish off that blog post series...
*Thread Reply:* const { defineStep } = require("@cucumber/cucumber"); const readline = require("readline");
defineStep('pause', {timeout: _1}, async function() { const rl = readline.createInterface({ input: process.stdin, output: process.stdout }); return new Promise((resolve) => { rl.question("Pause Step in Gherkin. Press Enter to Continue.", () => { rl.close(); resolve(); }); }); });
*Thread Reply:* So what this step does is pause cucumber. Useful for tests using Selenium or Chromium where you want to see what's going on in the browser, maybe click around a bit and check things.
Just re-posting this here in case anyone can help with a cucumber js specific issue.
*Thread Reply:* Are you validating that the db is cleared when the before step runs?
*Thread Reply:* @catharsisjelly maybe you're missing an await in your Before hook or something?
*Thread Reply:* @Cat No I assumed it was done, I'm used to PHP more so a few JS things are tripping me up
*Thread Reply:* @matt-SmartBear My Before function hook looks like this if it helps
Before(function () {
for (let collection in dbConnection.collections) {
dbConnection.collection(collection).drop()
.catch((e) => {});
}
});
*Thread Reply:* I figured dropping the collections was the fastest way to truncate them
*Thread Reply:* Do you need to await the call to drop()? Does it return a promise?
*Thread Reply:* it does, I have tried it with an await too and still errors.
*Thread Reply:* My guess is that's what's going wrong - it's a timing issue that means Cucumber has started the scenario before the drop() promise has resolved.
*Thread Reply:* Can you show us what it looks like when you use a promise?
*Thread Reply:* Yeah I was thinking the same thing... You want to see the output or code?
*Thread Reply:* Oh.. maybe jest is trying to run 2 features concurrently...
*Thread Reply:* If each feature runs fine all by itself that makes sense that itt might not if the second feature is running at the same time
*Thread Reply:* Yeah that will definitely cause yo problems if there is shared state
*Thread Reply:* Not sure how to set that up using jest and cucumber-jest but I'll dig through some docs. But for now here is that Before function with the await
Before(async function () {
for (let collection in dbConnection.collections) {
try {
await dbConnection.collection(collection).drop();
} catch (err) {
// collection does not exists, so iit does not matter that it was not dropped
}
}
});
*Thread Reply:* That looks good to me. That should at least mean that two consecutive scenarios don't clash in the database. If you have concurrent scenarios running though, that's a different problem.
FWIW I think you could also use map to turn the collections array into an array of drop promises, and then return Promise.all of that, to make your code more concise.
*Thread Reply:* It is possible to keep running scenarios concurrently, but you'll need to make sure everything going into the database has a unique ID,
*Thread Reply:* think about each scenario as a user working on your app - if you want them to be able to hit the app in parallel, you need to partition their data.
*Thread Reply:* Yeah that helps thanks @matt-SmartBear It's slowly dawning what might be the issue. No idea how to fix iit yet but thats half the issue.
*Thread Reply:* @matt-SmartBear for the map thing, do you have any docs links I could read up on how that works, it's not a JS concept I'm aware of
*Thread Reply:* Not sure of a good link for this. Here's how I think the code might look:
Before(
() => Promise.all(
dbConnection.collections.map(
collection => dbConnection.collection(collection).drop()
)
)
)
*Thread Reply:* Essentially, if drop() returns a promise, you can use map to turn the array of collections in to an array of promises to drop those collections. Then you can use Promise.all to turn that into a single promise that all of the collections have been dropped. Then you can give that single promise to Cucumber as the thing you want it to do before it runs a scenario.
*Thread Reply:* Yeah just reading the Mozilla docs on Promise.all
*Thread Reply:* I have some code that wraps around cucumber - it starts go hugo server and cucumber in child processes and shuts down the hugo server when the cucumber thread finishes. It also has Puppeteer js getting started and stopped and an API mocking server built on express. It's solid, but a bit complex and it has a few command line flags. I am currently writing cucumber tests of those flag using root cucumber - and it's working so I have the situation of a cucumber running in a cucumber.
What is that called though? 🙂
*Thread Reply:* Joking aside, my inner cucumber process isn't emitting any output but the .'s. My console log statements are being lost.
*Thread Reply:* We do this: https://github.com/cucumber/cucumber-js/blob/main/features/support/world.ts#L53
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* It will take me awhile to parse through it and understand it, but yeah, looks like what I need.
*Thread Reply:* Took a couple days but this did work. The largest problem points to something that might be addressed in Cucumber core.
*Thread Reply:* The tester was thrown off by a parameter being '"@devices"'
*Thread Reply:* Perhaps the parsing of the tags arg should silently toss extra quotes?
*Thread Reply:* A simple way is to have your page object classes take your World as an argument in their constructor. Then you can new them up from steps/hooks (or, manage the creation of page objects with your World itself)
*Thread Reply:* I’ve been testing in chromium. Not sure if you’re asking generally or want to hear from the dev team. I’m just a user.
*Thread Reply:* @matt-SmartBear You are right, actually it works. Only not work with the nightwatch.js commands. I will check that. Thanks!
*Thread Reply:* sounds good - been working on https://github.com/cucumber/cucumber-js/pull/1649 so interested to compare
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I'll look at that branch in detail tonight after work.
*Thread Reply:* At a glance though that functionality will let me simplify my code stack further.
*Thread Reply:* Cause for the most part I'm running cucumber in a child process using node -r esm path/to/cucumber/
*Thread Reply:* I've never noticed that feature
Which CI are you using? If using github action, which action are you using?
*Thread Reply:* [It is and has been a github workflow.] The only external actions I’m using, or have ever used, before the tests are:
uses: actions/checkout@v2
uses: actions/setup-node@v1
*Thread Reply:* [We are making] a framework that uses cucumberjs for some of its functionality. The start of the script in package.json is ./node_modules/.bin/cucumber-js --require ./lib/index.js. @Aurélien Reeves
*Thread Reply:* When do you say that it showed the progress of the steps, what do you mean exactly? Would you have a screenshot?
*Thread Reply:* I don’t have a screenshot, sorry. Didn’t know I’d need it 😉 You can see it in your terminal when you run it locally, though. As I said, it’s the little progress dots that appear - one for each step.
*Thread Reply:* ☝️ @Aurélien Reeves. Sorry for the late reply. Slack doesn’t color the dot red if my username isn’t used.
*Thread Reply:* @plocket are you talking about the progress formatter which shows dots on the console when executing tests? The thing that looks like the following:
```$ node ./bin/cucumber-js --format=progress ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
264 scenarios (264 passed) 1356 steps (1356 passed) 0m45.247s (executing steps: 0m33.205s)``` And they have disappear from your github workflow logs? In some place like on the screenshot attached?
*Thread Reply:* Exactly, @Aurélien Reeves! That’s exactly it. I no longer see the dots on the github workflow logs.
*Thread Reply:* And thanks for the ping. It’s been a crazy day and I wouldn’t have noticed at all.
*Thread Reply:* Those dots are written to the console with the progress formatter.
Check the formatter(s) you are using within your workflow. If you want the dots back, make sure to add the progress formatter.
You can do that with command line arguments, but also with profiles.
Documentation for the profiles: https://github.com/cucumber/cucumber-js/blob/main/docs/profiles.md
Note: the progress formatter is not the same one than the progress-bar formatter.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I haven’t had a chance to look into this, but I did find that when I inserted console logs in the middle of the steps, the progress dots would appear as soon as the first console log happened. It makes me think this is a setting in github. Something about log levels or something. Maybe process.stdout.write doesn’t appear during the test run, while console.log does for some reason.
*Thread Reply:* Do you want to show the progress from the github action in real-time? Do you have any issue with the progress formatter when executing the tests on your own computer?
*Thread Reply:* I do want to show the progress in real time. Local tests show progress just fine.
*Thread Reply:* Ok, thanks for the explanation.
This seems related to github, not to cucumber, specially if you are not encountering issues locally.
I guess github action logs are streamed one line at a time, when a new-line character is encountered. The progress formatter does not put new-line characters, so github action logs are not streamed in realtime.
You see some progress when you add some console.log because it will put a new-line character
*Thread Reply:* You may consider using your own custom formatter. You may override the buil-in progress formatter and add new-line characters in addition to the dots. That should trigger the stream from github in "real-time"
*Thread Reply:* We’ll put that in the issue to try. Does your GitHub show the progress?
*Thread Reply:* As seen on the screenshot from last time, yes, it does
*Thread Reply:* Ah, ok. That looked to me like progress dots after everything was done, not while the workflow was running. 👍
*Thread Reply:* Oh, you meant the progress!? I guess not. As I said, GitHub shows logs per line, when it detect a new line character
*Thread Reply:* Ok. Yeah, the new line thing happens to you too. That seems like it’s related to cucumber as well as GitHub, then, since cucumber isn’t putting a new line at the beginning of logging progress.
*Thread Reply:* I think there won't be any retry after one success. So you won't have more than one success.
For example if you have --retry 10, one test fails on the first attempt, if it succeed at first retry, it won't retry anymore.
*Thread Reply:* Thanks @Aurélien Reeves . Sorry for the confusion, what I mean should be first two tries failed, and third passed.
*Thread Reply:* If that 1/3 pass success rate can be reported, it will help us to find out flaky issues.
*Thread Reply:* Or at least the number of failed attempt
That is interesting. I suggest you submit a feature request: https://github.com/cucumber/cucumber-js/issues/new?assignees=&labels=&template=feature_request.md&title= 🙂
*Thread Reply:* If it’s not a feature yet, happy to submit the request. Thanks @Aurélien Reeves!
*Thread Reply:* I am not sure actually, but I don't think it is a feature yet.
And it may depends also on which formatter you may request that feature. Some formatters are commonly used by severals implementation of cucumber, but the retry mechanism is not, so there may be interesting incoming discussion 😛
That would be easier to follow on a github issue than in a thread on slack 🙂
*Thread Reply:* FYI you're not using the default html formatter, you're using something like a jenkins one, so you'd need to go there for that.
*Thread Reply:* Hi @luke that image is just from the article I linked. I am using https://github.com/gkushang/cucumber-html-reporter and checked in https://reports.cucumber.io/. Didn’t see the retried details.
*Thread Reply:* Do you mean it’s supported in cucumber default HTML formatter?
*Thread Reply:* I just tried html formatter and I couldn’t find retry relevant information.
*Thread Reply:* https://github.com/cucumber/cucumber-js/issues/1739 For further discussion.
*Thread Reply:* Thanks @Tim Yao 👍
As you noticed: this request is related to formatters.
On Cucumber we may address some of our own ones, but when using other formatters like the one from gkushang, you should also ask them too.
*Thread Reply:* Yes. gkushang one is based on the deprecated json formatter, so I think I may switch to core html-formatter in the future.
*Thread Reply:* If there are features in the gkushang one you like that we're not implementing yet, we'd love to hear about it!
*Thread Reply:* Thanks @matt-SmartBear will try and see what is the difference
*Thread Reply:* Do you want to execute the test of the playwright.js repository?
*Thread Reply:* If so, try npm test
To execute the tests you are setting-up using cucumber and playwright.js, execute cucumber as usual
*Thread Reply:* Hi @LayMui don't get fooled by the name - it has nothing to do with the Microsoft Playwright library. See https://github.com/cucumber/playwright.js/issues/31 - we might rename it back to @cucumber/screenplay.js
<a href="https://github.com/cucumber/playwright.js">cucumber/playwright.js</a>
*Thread Reply:* maybe https://github.com/cucumber/cucumber-js/blob/main/CONTRIBUTING.md is what you are looking for?
*Thread Reply:* Great news @Michael Morris!
Sadly we do not have a great contributor/developer documentation yet 😞
There is an on-going effort to improve documentation for new contributors. At the moment it was focused on cucumber-ruby and on the "welcoming" documents like README and CONTRIBUTING.
We have an issue to exchange ideas on that topic: https://github.com/cucumber/common/issues/1521
What kind of documentation were you looking for? Would you have some links to existing ones in other OSS projects as reference?
You can submit your suggestions, ideas and request in the issue, that will be easier to keep track later 👍
<a href="https://github.com/cucumber/common">cucumber/common</a>
*Thread Reply:* That's fantastic @Michael Morris!
One of our goals for the year is to increase the number of new contributors, so anything you find difficult, confusing, or even just a little bit annoying as you take your first steps, we'd love to hear about it.
I'm available for 1:1 mentoring if you'd like some pairing time. https://calendly.com/mattwynne
*Thread Reply:* FWIW, the place to start contributing for any github repo should be https://github.com/cucumber/cucumber-js/contribute
That page will show the contributing guide, and any issues tagged with "good first issue".
*Thread Reply:* If other users are looking into this, we have discussed this issue here: https://github.com/cucumber/cucumber-js/issues/1741
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I need it to be set during the first test and remain afterwards
*Thread Reply:* I am using nightwatch.js with cucumber.js, I use nightwatch globals to manage state.
*Thread Reply:* Looks like a selenium alternative, not something I'm looking for ATM.
My best thought so far is to disable this new setup stuff until we've integrated the work being done in the pre-test script so that it can be run on demand, so we no longer run into this limitation.
Which means the recent work needs to be left on hold.
😞
*Thread Reply:* Doesn't work: https://github.com/cucumber/cucumber-js/issues/1393
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* It will only work with Before hook, hence "per-test", which won't work with our current setup.
*Thread Reply:* In my use case, I have a global vars like test id, project name , also have some initialisation data as states to share across all tests.
*Thread Reply:* I also set test id to env var, so I can share the id across CI jobs.
*Thread Reply:* like multiple cucumber process can share the same test id.
*Thread Reply:* no, if I use env var, it’s set outside cucumber, like in CI job.
*Thread Reply:* Right, I'm looking for a way to • set a variable inside a beforeAll hook • have it carry over to next tests
*Thread Reply:* yep, that's exactly my issue, it won't carry to the next scenario
*Thread Reply:* this, inside a cucumber scenario, or even hooks usually refers to the world object, does it not?
*Thread Reply:* I haven’t use world yet, looks like this is point to world .
*Thread Reply:* By using global state, will it be a issue when we run cucumber parallel?
*Thread Reply:* I just have to find the JS alternative to boto3 / awssrp, something I should have done a long time ago 🤷
*Thread Reply:* ideally we need to do node js stateless way. as it’s not work for multiple node process.
*Thread Reply:* I see https://github.com/cucumber/cucumber-js/blob/main/docs/parallel.md cucumber-js --parallel 3 only shares env vars, not in memory vars, that can cause you lost the global state. My nightwatch global won’t work in parallel I guess. 🙃
*Thread Reply:* To solve that problem, need to put state management into a shared db.
*Thread Reply:* boto3 / awssrp is for storing state in AWS s3? that sounds a good solution for me.
*Thread Reply:* @Mona Ghassemi what do you think to the proposal in the issue you linked? ie to allow some state to be set in a BeforeAll which you can then read (not write) from scenarios - would that solve your need?
*Thread Reply:* @Mona Ghassemi you should be able to use process.env.SOME_ENV_VAR = "a-value" in your beforeAll hook, and then read that later on, no?
*Thread Reply:* I’m surprised nobody has suggested using global variables yet. That’s a JavaScript thing, not a Cucumber thing.
*Thread Reply:* @Aslak Hellesøy I think you are talking about using global object like global.myTestState . I tested it, works. I think it’s same as my Nightwatch.js globals data. However, I guess it may not work in parallel. As it’s in the memory, not able to be sync between parallel processes. Haven’t test it yet.
*Thread Reply:* Can you share more about what you've tried, and what specific errors or other problems you've encountered?
*Thread Reply:* Yep it’s per scenario - once we get to actually running tests, which feature file it’s in doesn’t mean much.
*Thread Reply:* If you’re interested, someone is working on a way to control the allocation of scenarios to workers.
*Thread Reply:* https://github.com/cucumber/cucumber-js/pull/1588
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I have a feeling it's not super straightforward as that message "View your Cucumber Report at:..." is sent back verbatim from the reports server, so there's no structured data with the URL in it. I night be wrong though, but I think I've asked the same question before.
*Thread Reply:* Thanks! If all else fails I think the old fashioned xml should be usable. Was hoping it would be easy-peasy-lemon-squeezy 🍋
*Thread Reply:* We have an open issue about this for Cucumber-JVM: https://github.com/cucumber/cucumber-jvm/issues/2324
If you’re using a different Cucumber implementation, please raise a corresponding issue.
<a href="https://github.com/cucumber/cucumber-jvm">cucumber/cucumber-jvm</a>
*Thread Reply:* FYI: Cucumber issues a GET request first. The server responds with 204 and an upload URL in the Location response header, plus a plain text banner that cucumber prints after PUT ing the cucumber messages to the «pseudo redirect» URL.
You can find a bash implementation of the upload client here: https://github.com/cucumber/common/blob/main/scripts/publish-cucumber-report
<a href="https://github.com/cucumber/common">cucumber/common</a>
*Thread Reply:* Thanks @Aslak Hellesøy! Yes, am using the js implementation so I guess I'll need to raise the issue. Will probably look into these internals a bit too when I get a chance!
*Thread Reply:* It might be worth raising an issue here https://github.com/TheBrainFamily/cypress-cucumber-preprocessor - AFAIK that plugin only uses very little cucumber stuff - just the Gherkin parser.
*Thread Reply:* @David Goss I understand but my question was about if there is an alternative to cypress-cucmber-preprocessor
*Thread Reply:* Yes the formatters run on the coordinator and all the scenarios are running on the worker. You could this.log the information from a step or hook, which will emit an attachment. Also you might be interested in https://github.com/cucumber/common/issues/1317 which is about being able to emit custom stuff in the meta message, although again this would happen in the coordinator
<a href="https://github.com/cucumber/common">cucumber/common</a>
*Thread Reply:* Ah. Will try that, I think I was trying to do it like
@nightly, not @notReady
*Thread Reply:* Tag expressions are infix boolean expressions. There are no commas or other punctuation. https://cucumber.io/docs/cucumber/api/#tag-expressions
*Thread Reply:* Specifying only @nightly would be sufficient, unless you have scenarios tagged with both @nightly and @notReady. Why would you tag a @notReady scenario with @nightly?
*Thread Reply:* It's just a toy example, I don't have a @notReady tag.
*Thread Reply:* I didn't think through the example very hard but I do have cases where I want to exclude something 😄
*Thread Reply:* I don't have the history of that, just a personal though: that could avoid forgetting about pending things
As a developer, at some point, I could totally forget about pending things, specially if they are totally ignored. However if they are responsible for failing my CI for example, I cannot ignore those
*Thread Reply:* Thanks @Aurélien Reeves that makes sense. In my case, I want to leave some test cases skipped based on the environment. For example, one test case works in most of browsers but not IE11, I may return pending for IE now and fix it in the future. However, I wish it can still pass CI in IE with this pending step.
*Thread Reply:* In your case, would "skipped" be more appropriate than "pending"?
*Thread Reply:* It is just because I don’t know I can return skipped … looking at the docs now
*Thread Reply:* Works like a charm! Thanks @Aurélien Reeves I will stick with skipped then.
*Thread Reply:* also for pending steps you can also tag the feature not to run (Which is probably better behaviour), it's what we do.
*Thread Reply:* @luke I do has tags like @skip then use it in cli tags like not @skip . But I see someone do it like this:
Before({tags: "@skip"} async () => {
return 'skipped';
});
So I don’t need to add not @skip to cucumber tags argument, is that a good practice?
*Thread Reply:* So there's a difference here, so if you do it that way, it'll return a cucumber skipped exception.
Now in my eyes, that is ok. So something then causing that to fail I wouldn't support. But I'm 99% certain that they aren't prohibited, just not defaulted.
*Thread Reply:* Because doing it dynamically on the fly also allows you to configure it based on params. Like we have a whole class that determines whether to skip a scenario.
*Thread Reply:* I get it'll return a cucumber skipped exception is the difference.
*Thread Reply:* And second way is dynamic may give me some flexibilities.
*Thread Reply:* irrelevant question maybe, how do you disable tests failed because defect?
*Thread Reply:* I tried to tag it @defect then use not @defect in command to ignore it.
*Thread Reply:* That looks like something very specific to your need, so using tags as you did looks good to me
*Thread Reply:* that tag does nothing. But we know every tag that has failing SHOULD have a ticket number assigned as well.
*Thread Reply:* I use a comment for that purpose. but tag sounds a better way.
*Thread Reply:* You can do that exactly as you would with cucumber. For example you can use command line arguments: https://github.com/cucumber/cucumber-js/blob/main/docs/cli.md#tags
Or you could also use specific profiles: https://github.com/cucumber/cucumber-js/blob/main/docs/profiles.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Using profiles
Or updating package.json to add your filter into the "test" script
Or you don't use npm run test, but npx cucumber-js with the --tag argument
*Thread Reply:* We use something like this:
node ./node_modules/.bin/cucumber-js --tags='@tagName'
*Thread Reply:* make it shorter with npx cucumber-js --tags-"@tagName"
*Thread Reply:* We use named scripts in the package.json btw, so like: scripts: { testWeb: node ./node_modules/.bin/cucumber-js --tags='@web', ... }
and then at command line:
yarn run testWeb
(or npm if that's what you use).
*Thread Reply:* I think I realized that my issue was that we use dotenv, but to be perfectly honest, I think I can remove it as the way we need to import variables per environment probably makes it redundant so we have ""node -r dotenv/config ./node_modules/.bin/cucumber-js --""
I will probably look at removing dotenv
*Thread Reply:* TSError: ⨯ Unable to compile TypeScript: src/steps/playwright.steps.ts:2:23 - error TS6133: ‘DataTable’ is declared but its value is never read.
2 import { Given, When, DataTable } from ‘@cucumber/cucumber’;
*Thread Reply:* my feature has datatable like this
Scenario: Fill up registration form
When he fill up his registration details
| username | age | residency | accounts | nationality |
| johnmark | 48 | resident | Savings | USA |
Then he is able to submit his form
*Thread Reply:* but it seem to affect the original step defn by you
@foo
Scenario: Change theme # features/playwright.feature:7
Given Go to the playwright website
? undefined
When Change theme to "light" mode
? undefined
Then Snapshot
- skipped
When Change theme to "dark" mode
? undefined
Then Snapshot
- skipped
*Thread Reply:* The low-tech way would be search for the step def in the repository and see if you get other matches (depending on what text editor / IDE you have, there is generally a way to do it).
*Thread Reply:* this is very low-tech.. must be there’s a better option
*Thread Reply:* Yeah I understand if you have a lot of features / code it's not ideal! But now I'm curious too. 🙂
*Thread Reply:* ~Maybe with some code coverage tool?~
Found that the usage formatter actually detect unused step definitsions
*Thread Reply:* The usage formatter - built-in with cucumber-js - is actually able to report unused step definitions.
https://github.com/cucumber/cucumber-js/blob/main/src/formatter/usage_formatter_spec.ts#L33
https://github.com/cucumber/cucumber-js/blob/main/features/usage_formatter.feature#L8
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* it seems a good solution. Is there documentation for its usage from Docs or readme file?
*Thread Reply:* Formatter docs are a bit terse but I have some improvements in flight, see https://github.com/cucumber/cucumber-js/blob/docs-rework-formatters/docs/formatters.md#usage
*Thread Reply:* You probably want to use “dry run” at the same time so it doesn’t have to actually execute all your tests to find the usage
*Thread Reply:* This can be achieved by storing locally what you need with localStorage. See https://github.com/lmaccherone/node-localstorage if you use Node which looks like it’s the case for you.
So, in your first step definition, you would store each key/value pair in a local file using setItem(key, value). The name of the file is the key and the content of the file is the value (as a string).
You can then access those key/value pairs in subsequent steps with getItem(key).
And you can delete all localStorage files as part of your After(Scenario) hook.
Note that everything in localStorage is saved as a string. So you might have to use JSON.parse whenever you need to access an object, an array, a boolean etc.
*Thread Reply:* You can use the world: https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/world.md
You can assign your datatable to the wold in your first step, then use it in further steps
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* It looks like you're using a screenplay pattern implementation. Nice!
If you happen to be using https://github.com/cucumber/screenplay.js/ you can do it like this: https://github.com/cucumber/screenplay.js/#sharing-data-between-steps
If not, perhaps you can implement something similar in your implementation of the actor.
*Thread Reply:* This is a timely question! I’m currently working on https://github.com/cucumber/cucumber-js/issues/1711 to add a composable API to enable things like this. Would you mind dropping a comment on there with this use case and I’ll try and make sure it’s covered?
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* looks like a great addition, would this allow to easily pipe the scenarios in or does it just allow to programatically run it giving it the names of feature files?
*Thread Reply:* Sorry for late reply. There'll be a PR after this weekend. First pass might be simpler initially but the goal is to have it composable so one could e.g. pass in scenarios like you say. I'll keep you posted.
*Thread Reply:* @Vincent Pretre is this a case for using gherkin-utils?
*Thread Reply:* I have the same problem in the afterStep hook. I have a testStepId but i can’t see any way to get the step keyword or text using that because in the hooks i don’t have access to the EventDataCollector.
*Thread Reply:* In the changelog: https://github.com/cucumber/cucumber-js/blob/main/CHANGELOG.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Thanks. I'm an idiot though, I asked this question then it immediately occurred to me to google for cucumber.js 7.3.1 changelong which gave your link as the 1st result 🙂
*Thread Reply:* ESM support is in core. Ok, I can rework my code to stop using the esm package.
*Thread Reply:* We removed it in 7.2.1 - the change in 7.2.0 did not work. So 7.3.1 has no esm support AFAIK. People are still working on it.
*Thread Reply:* I upgraded because Cucumber isn't reading it's profile or tag arguments off the command line.
*Thread Reply:* See, I have a script that reads it's own arguments off the command line to config some things, then passes off control to cucumber.
*Thread Reply:* Not that I know of @Michael Morris - got an example?
*Thread Reply:* I'm ok now - The problem lay in how golang mage relays arguments. Cucumber is just fine and is the least of my worries today.
*Thread Reply:* Could you please give more info?
There is actually an issue that may be related: https://github.com/cucumber/cucumber-js/issues/1796#issuecomment-926318483
Do you experience the same?
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* As you can see, once everything is up-to-date, I was not able to reproduce the issue.
Would you have the possibility to provide a repo with a minimal reproduction example, and all the info regarding your execution environment? Versions of cucumber-js, node, and npm, os, and the shell?
*Thread Reply:* I have create a file .env to set all my environment variables for staging. example: FARMERBASEURL=“https://farmer.stage.xx.io
*Thread Reply:* I want to set one for preprod FARMERBASEURL=“https://farmer.preprod.xx.io
*Thread Reply:* in my package.json, “test:execute”: “cucumber-js --tags=‘@test’“,
*Thread Reply:* Did you required the "dotenv" package as part of your support code?
*Thread Reply:* yes in my step defn, I add this
const dotenv = require('dotenv').config();
*Thread Reply:* then I call
ChangeApiConfig.setUrlTo(process.env.FARMER_SERVICE_BASE_URL)
*Thread Reply:* what I need to be able to switch between 2 env. one for staging one for preprod
*Thread Reply:* It seems this is not related to cucumber 🤔
What do you want to accomplish exactly?
Would you like to execute cucumber tests on both environment? Or do you want to execute the tests twice on the same environment but simulating different environments? Or something else?
*Thread Reply:* I do have the dotenv package but mostly I have ended up sourcing from different files, so it might be redundant at this point. It has nothing to do with cucumber; cucumber doesn't manage environment variables.
*Thread Reply:* ok thanks I got it resolved. using the NODE_ENV=stage cucumber.js …
*Thread Reply:* I guess you are using cucumber expressions The repo is there: https://github.com/cucumber/cucumber-expressions
You'll find some documentation, and a sandbox to test your expressions
*Thread Reply:* here is my feature file ```@all Feature: Get farmer details In order to get farmer details As a API farmer services consumer James James wants to get farmer details based on UserID Background: Given James is at the farmer services url
Rule: Get farmer detail using a valid unique UserID
Scenario: Able to get farmer detail using userID When he get a farmer using a UserID Then he should able to get the details of the farmer
Rule: Get farmer detail using an invalid unique UserID @bug Scenario: Able to get farmer detail using userID When he get a farmer using a UserID Then he should able to get the details of the farmer```
*Thread Reply:* when running the test, I have this line in my package.json
NODE_ENV=stage cucumber-js --tags=all
*Thread Reply:* ```Rule: Unable to delete farmer from the database with invalid UserID
@test
Scenario: Unable to delete farmer from database with invalid UserID
When he delete a farmer with an invalid UserID
| invalidUserID |
| 9ebea001-89df |
| 89df |
Then he should be unable to delete the farmer
| statusCode | errorMessage |
| 400 | Path variable 9ebea001-89df is not a valid UUID |
| 400 | Path variable 89df is not a valid UUID |```
*Thread Reply:* that is the first row of the When steps will map to the first row of the Then steps, and so forth?
*Thread Reply:* I think an example table would be more appropriate: https://cucumber.io/docs/gherkin/reference/#scenario-outline
```Scenario Outline: Unable to delete farmer from database with invalid UserID
When he delete a farmer with <invalidUserId>
Then he should be unable to delete the farmer
And he shoud see <errorMessage>
Examples:
| invalidUserId | statusCode | errorMessage |
| 89df | 400 | not valid |```
Using datatables, you would have to keep the mapping between your steps by yourself.
*Thread Reply:* ok so the example I gave should still work as long as I have 2 rows for When and 2 corresponding rows for the Then steps? cos in case I have alot of parameter, I want to keep the When statement concise and brief
*Thread Reply:* It would work. The order specified in the gherkin scenarios for datatables should be the same order in your step definitions
*Thread Reply:* You could try https://school.cucumber.io to learn the basics
*Thread Reply:* https://github.com/smartBear/git-en-boite is a repo I worked on recently that uses Cucumber JS to drive the implementation. It's an HTTP service though, not a web app.
*Thread Reply:* Feature files: https://github.com/SmartBear/git-en-boite/tree/main/doc/features
Automation code: https://github.com/SmartBear/git-en-boite/tree/main/packages/acceptance-tests/src
*Thread Reply:* You can search GitHub for projects with Gherkin in them, not sure if / how you refine that just to TypeScript/JavaScript projects.
https://github.com/search?l=&q=language%3AGherkin&type=repositories
*Thread Reply:* @Aslak Hellesøy I have read a lot docs and tutorials(include https://school.cucumber.io) also used in my project. But feel most of the tutorials are too simple, like a hello world . When I start use it in a big scale project, I got too many questions.
*Thread Reply:* Ask the questions @Tim Yao! Then come back and answer other peoples' when they have them later!
*Thread Reply:* Thanks @matt-SmartBear for the sharing. The search language is a good idea, I will use it to explore other repos. 😁
*Thread Reply:* Some opensource has a awesome … repo, list all projects used the product.
*Thread Reply:* Yeah that would be good. It's always been quite rare finding open source projects that use Cucumber - so many are proprietary and in-house, but they are out there
*Thread Reply:* I guess maybe there is already stuff like this in the docs, probably hideously out of date...
*Thread Reply:* If you want me to create a repo under the cucumber org for this, lmk
*Thread Reply:* For me, I think cucumber has a nice documentation site already, so mainly is for other resources.
*Thread Reply:* I also try to find time read your blog https://cucumber.io/blog/. It’s good place to learn as well.
*Thread Reply:* Please let me know when you got the repo set up 🙂 cheers
*Thread Reply:* Let's see if this trawls anything up https://twitter.com/mattwynne/status/1452515301396922368
Twitter
*Thread Reply:* That's a WebDriver error.
https://developer.mozilla.org/en-US/docs/Web/WebDriver/Errors/StaleElementReference
It happens when you have a handle to an element, but by the time you run the code to interact with it, it's been deleted from the page.
*Thread Reply:* It sounds plausible that you can't call it more than once. What's the error that you see?
*Thread Reply:* I will try "if(success)" https://cucumber.io/docs/cucumber/api/
*Thread Reply:* not sure what it's called in JS. It occurs after each scenario.
*Thread Reply:* Ok. But what I need to know is how do I interrogate the state?
Cucumber adds green checkmarks to steps that passed. Red x to the ones that failed, etc.
So there must be some way to access the pass / failed state?
*Thread Reply:* Ok maybe this will help https://cucumber.io/docs/guides/browser-automation/#screenshot-on-failure
*Thread Reply:* I tried with
if (scenario.isFailed()) {
however I get this error:
[0-0] 2021-10-29T16:58:23.360Z ERROR @wdio/utils:shim: TypeError: scenario.isFailed is not a function
*Thread Reply:* afterScenario: function (scenario, world) {
if(scenario.isFailed()) {
...
}
*Thread Reply:* Ok I realize it might be more WebdriverIO that I need to deal with https://webdriver.io/docs/configurationfile/
Looks like there is an optional "result" param; will try that
afterScenario: function (world, result) {
See here for more info / discussion (Different flavour, but same principle)
https://github.com/cucumber/cucumber-ruby/issues/1432
<a href="https://github.com/cucumber/cucumber-ruby">cucumber/cucumber-ruby</a>
The JS variety is here: https://github.com/cucumber/cucumber-js/issues/1740
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
Well it looks from https://github.com/cucumber/cucumber-js/pull/1775 like there might be some light at the end of the tunnel.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
Are you able to try the v8 release candidate? https://github.com/cucumber/cucumber-js/blob/main/CHANGELOG.md#800-rc1---2021-10-19
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* ```Feature: Button Event Handling In order to validate Button event handling As a QA tester Jan Jan wants to see how the ahua button event behave
Background:
Given Jan is at the app
Scenario Outline: Ahua Button Event Change title When he want to trigger button event to change the button title | event | | onPress | Then he is able to see the button title changed "<title>" Examples: | title | | buttonTitle |```
*Thread Reply:* You can use both scenario outline and datatable in the same scenario.
But they won't be automatically "combined" as you would like as I presume from your sample. You would have to program it by yourself.
However, in your case, why couldn't your writte the following?
```Feature: Button Event Handling In order to validate Button event handling As a QA tester Jan Jan wants to see how the ahua button event behave
Background:
Given Jan is at the app
Scenario Outline: Ahua Button Event Change title When he want to trigger button <event> to change the button title Then he is able to see the button title changed "<title>" Examples: | event | title | | onPress | buttonTitle |```
*Thread Reply:* A test case is the result of a scenario being transformed as a test. A scenario with N examples will result in N test cases. The test case is an example.
The Before hook is executed before each test case, not before each scenario.
Thus the before hook has no access to other examples of the scenario than the one it has been calling for.
Does that help?
*Thread Reply:* @Aurélien Reeves thanks for the explanation. Yeah, I get that, but I find it a bit tricky to get the "examples" for current test case.
Currently exploring the astNodeIds to match them with ids in gherkinDocument
*Thread Reply:* What do you call "examples" for the current test case?
*Thread Reply:* There are no "examples" for a test case: a test case results from a single example of the scenario outline
*Thread Reply:* Ok, I am trying to get the "examples" inputs (the table data) for each test case as I have a logger and need to distinguish the different test runs of the same scenario outline.
*Thread Reply:* made it work like this: ```const parseExamples = (testCase: ITestCaseHookParameter) => { const examplesMap = new Map() const scenarioId = testCase.pickle.astNodeIds[0] const exampleId = testCase.pickle.astNodeIds[1] const scenario = testCase.gherkinDocument.feature.children.find((child) => child.scenario.id === scenarioId) const cells = scenario.scenario.examples[0].tableHeader.cells const values = scenario.scenario.examples[0].tableBody.find((body) => body.id === exampleId)
cells.forEach((cell, index) => {
examplesSet.set(cell.value, values.cells[index].value)
})
return examplesMap
}``
but not sure if the order ofastNodeIds` is guaranteed and if there could be more ids in some cases?
*Thread Reply:* I see. You want the row in the table which has been used to generate the test case? Not all the rows?
I guess your snippet should work well indeed
*Thread Reply:* Yes, thats it, sorry for not using the exact terms 😉 It works fine it seems , it was just missing a couple of optional chaining to avoid failures for non-scenario outlines.
*Thread Reply:* @matt-SmartBear it's just not super user friendly, took me a while to comprehend it.
*Thread Reply:* In Ruby, I think we wrap this raw test case object with a facade that has some more helpful stuff on it. We could consider doing that for JS too.
*Thread Reply:* @Luděk Nový what would the API have looked like here, in an ideal world? Could you sketch out some code?
*Thread Reply:* Thats a tough question, but for the above described use case, ideally it would not be necessary to iterate so many times, eg, exposing the example values in pickle object or something. and I did not find any docs for it, eg whether the order of astNodeIds is guaranteed, of there could be more then 2 items in the array and so on.
*Thread Reply:* So something like this perhaps?
Before((scenario: RunningScenario) => {
console.log(scenario.exampleRow?.cells)
})
*Thread Reply:* yes! that would be much easier to use than the current api 🙂
*Thread Reply:* OK I raised an issue so we can start the process of making this happen: https://github.com/cucumber/cucumber-js/issues/1851
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* @Luděk Nový if you'd be interested in working on a contribution for it I'd be happy to support you with it.
*Thread Reply:* It should be a fairly straightforward change, and super useful
*Thread Reply:* This works but it's ugly: "node -r esm node_modules/@cucumber/cucumber/bin/cucumber-js"
*Thread Reply:* And I'm not sure it will work with parallel mode.
@Michael Morris there is actually nothing to do to activate esm modules. It should just work out of the box within a project which uses ESM.
I've just pushed an example here: https://github.com/aurelien-reeves/cucumber-js-esm-example The project is a node module using ESM. The support code also uses ESM without any specific configuration. Please just note the use of a common-js configuration file: this is a current limitation of cucumber-js as explained at the bottom of the following document: https://github.com/cucumber/cucumber-js/blob/main/docs/esm.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Actually your cucumber.js file is working as a module because of your hack.
Your hack load cucumber itself as a module, which it is not. I would encourage you to try removing your hack and using a .cjs file for the configuration. Everything else should work properly using esm imports / exports.
*Thread Reply:* They don't though. I tried that first - path of least resistance and all.
*Thread Reply:* I'm in an odd situation. I have a client extending library of cucumber that adds API services emulation for tests that is imported as a module by the actual projects that have the critical system tests.
*Thread Reply:* The reason for this structure is there are multiple projects that need to import this library.
*Thread Reply:* Those projects exist as esm modules for... reasons. (Testing did not exist at the beginning of the project and I'm struggling to bring it in).
*Thread Reply:* I could extract my hack out to a tiny project - cucumber-esm. The ESM package smooths over a lot of node's mistakes with ES6.
*Thread Reply:* I am not sure to get it actually 😓
Do you think you could setup a minimal reproducible example in a dedicated new repository?
That way we could have a deeper look inside the issue you are facing, that may help us improving esm support
*Thread Reply:* Amazing! Thanks!
And no worries for the delay :)
*Thread Reply:* Well, this is too specific, and requires your script hack to work. So we won't be able to merge this as-is, sorry.
*Thread Reply:* ```async function runCucumberTest(args, expectError = false) { return new Promise((resolve) => { const allArgs = ["run", "self", "--", "--site", webserverPath].concat(args); const runner = spawn( /^win/.test(process.platform) ? "npm.cmd" : "npm", allArgs, { // Uncomment next line to pass inner test's output to console. Do this // only when debugging this script. // stdio: "inherit", env: { ...process.env }, } );
runner.on("exit", (code) => {
if (expectError && code) {
resolve("pass");
} else if (expectError && !code) {
throw new Error("Expected test to fail and it did not");
} else if (!expectError && code) {
throw new Error("Test failed unexpectedly");
} else if (!expectError && !code) {
resolve("pass");
} else {
throw new Error("This code should be unreachable");
}
});
}); }```
*Thread Reply:* To be clear, the code above works fine - it's my current work around on the lack of a true runner setup.
*Thread Reply:* My world object has an AfterAll hook that scans the browser console for errors. If it finds any, it fails the test immediately.
*Thread Reply:* I need to test that hook and this is what I came up with to insure it would fail on cue.
*Thread Reply:* What do you mean by "cucumber logging"?
Did you noticed the following: https://github.com/cucumber/cucumber-js/blob/f37edaadbefea557d87ef5f19bc301a277ac4264/docs/support_files/attachments.md#logging?
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* It relies on the formatters to document their options.
For built-in formatters, the option that is well documented is the one to disable colors for outputs to the terminal.
There is also the customization of snippets which uses those options. This is documented here: https://github.com/cucumber/cucumber-js/blob/c3a33139366a205d12e4e707eead75ae2ae1d2d8/docs/snippets.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* You can create a cucumber.js file in the root folder of your project, next to your package.json file
*Thread Reply:* When I tried that it continued to print the message. Is there another step included?
*Thread Reply:* That is weird Can you paste the content of your cucumber.js file, and the script you use to run your tests ?
*Thread Reply:* I had a typo in cucumber -_- I’m sorry, that did work after all
*Thread Reply:* No worries! I am glad you made it work after all 😁
*Thread Reply:* May I ask: what are you trying to achieve?
I am not sure we can know tags on a scenario from within a step directly. You may have to implement a specific world to do that.
I don't know if that may help, but you may have access to those tags from beforeStep and afterStep hooks.
Documentation for the "world": https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/world.md
Documentation for hooks: https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/hooks.md#beforestep--afterstep
More documentation in the api reference: https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/api_reference.md#afterstepoptions-fn
*Thread Reply:* Best way (Using my cucumber analogy), is you create an info class (in js probably an object), that you inject at world creation time (So from your hooks), then access this object inside your step invocation
*Thread Reply:* [Sorry, don’t have time to look at the previous comment’s links.] We definitely do it in Before(), which comes just before the Scenario runs. Here’s some sample code:
Before(async (scenario) => {
let scenario_name = scenario.pickle.name;
}
*Thread Reply:* Hi 🙂
I guess you can have the structure as you want:
• move the app itself from src to src/app • edit ./cucumber-js.conf output_directory from ./features to ./src/test • update ./features/actionwords.js to properly require the app from ./src/app rather than ./src • update the call to cucumber-js to let him know where to find features and step definitions Beside that, may I ask:
• why are you using hps-cucumber-javascript? • did you noticed that it is set-up with a pretty old version of cucumber? I would encourage to update your fork to work with latest version of cucumber-js. That may certainly require a lot of fix and customization.
*Thread Reply:* Thank you for your reply! 🙂 I'm new with cucumber-js, I saw that template and I'm trying to see how it works.
*Thread Reply:* Are you also a user of cucumber studio or hiptest publisher?
*Thread Reply:* If you want to grab those pieces separately you'd need to write your own custom formatter.
*Thread Reply:* There's an example of that here: https://github.com/cucumber/cucumber-js/blob/main/features/custom_formatter.feature
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* If you have a look at the source for the ProgressFormatter (https://github.com/cucumber/cucumber-js/blob/main/features/custom_formatter.feature) you should get some clues.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Thanks, we’ll check it out and give it a shot. Seems more needs to go into it than we thought, so it may take us a little while, but thanks for the pointers 👍
*Thread Reply:* Hello Tim,
The zephyr products have integration with Cucumber.
Zephyr, among other, are Smartbear products. And as you may know, Cucumber is supported by Smartbear too.
We have dedicated teams dedicated on integrating BDD and Cucumber into Zephyr. That team is really close to us at Cucumber Open.
*Thread Reply:* Yes, I know, as they are all in Smartbear, I assume they can integrated better than others. Is there any code example or demo video for cucumber-js Zephyr integration? I only see one for Java https://github.com/SmartBear/zephyr-scale-cucumber-integration-example
*Thread Reply:* At first, which Zephyr are you talking about? Zephyr scale, squad or enterprise? Those are 3 different products actually.
Then I'll be able to look for someone to give you more accurate help 🙂
*Thread Reply:* Zephyr scale I think. But sad Zephyr is not even on our QA’s list 😅
*Thread Reply:* Hi Tim! So both Zephyr Scale and Squad propose an integration for cucumber. Both rely on a call to an api endpoint to upload your results. In scale it is here: https://support.smartbear.com/zephyr-scale-cloud/api-docs/#tag/Automations. You either use the tag as described in the example you mentioned, or Scale will create your test cases on the fly if they do not exist yet in the library. Squad behaves the same way: https://support.smartbear.com/zephyr-squad-cloud/docs/test-automation/index.html even tho it is a bit more complex to create the task.
*Thread Reply:* In scale, your scenario results will look like this: https://app.getbeamer.com/zephyrscale/en/better-visualization-of-your-cucumber-results
*Thread Reply:* Thanks @David Lechevalier, that’s really helpful. For API, is there an integration plugin/addon for cucumber-js to use directly? Or I need to write by my own.
*Thread Reply:* No direct integration. You need to to somethnig like this:
curl -X POST <https://api.zephyrscale.smartbear.com/v2/automations/executions/cucumber>\?projectKey\=${PROJECT_KEY}\&autoCreateTestCases\=true \
-H "Authorization: Bearer ${ZSCALE_TOKEN}" \
-F "file=@results.zip"
Woith the project key being the key of your jira project and the ZScale token being the authentication token you get from Zephyr Scale. The zip files include the results of your execution as json files
*Thread Reply:* Got it. Thanks. I will take some time to have a look at them. 👍
*Thread Reply:* Before({tags: "not @foo"}, function () {
// This hook will be executed before scenarios tagged with @foo
});
*Thread Reply:* I don't think the comment says what you want it to say
*Thread Reply:* ```Caused by: Error: Wait timed out after 5s
[test:stage:execute] at xx/node_modules/webdriverio/build/commands/browser/waitUntil.js:66:23
[test:stage:execute] at async Browser.wrapCommandFn (xxx/node_modules/@wdio/utils/build/shim.js:131:29)
[test:stage:execute] at async World.<anonymous> (/xxx/src/step-definitions/createCredential.steps.ts:```
*Thread Reply:* seems like a webdriver thing. I’m not familiar with webdriver, but I use Cypress and there is a custom config you can pass to extend the wait time
*Thread Reply:* yeah same in webdriver https://webdriver.io/docs/api/browser/waitUntil
*Thread Reply:* Hi @plocket I guess you want to run Cucumber-js in a programmatic way. I found an open issue https://github.com/cucumber/cucumber-js/issues/1711 and looks like they are working on it for v8 but no public API yet.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* @plocket Here is an example how Nightwatch.js v2 uses cucumber cli in their js file https://github.com/nightwatchjs/nightwatch/blob/main/lib/runner/test-runners/cucumber.js Not sure if it’s a best practice. But maybe can help you, just copy it.
*Thread Reply:* From the small amount of information this is an error from the driver, not the selector.
*Thread Reply:* I tried to add a new declaration for driver :
his.driver = new selenium.Builder().withCapabilities({
browserName: 'chrome',
javascriptEnabled: true,
acceptSslCerts: true,
chromeOptions: {
args: ['start-maximized', 'disable-extensions']
},
// path: chromedriver.path
}).build();
but it throws the same error. Is there a better way initialize the driver? In the other steps the wait and findElements work with no initialization for the driver
*Thread Reply:* What happens if you console.log(this.driver) right before the original code?
*Thread Reply:* Silly question, but are you using this inside of an arrow function eg: () => {/****code here****/}
*Thread Reply:* I'm getting an undefined for the driver with console.log
*Thread Reply:* This is the function : checkEmptyEmailAndPassword: function () {
driver.wait(until.elementLocated(this.elements.emailPlaceHolder));
driver.findElements(this.elements.emailPlaceHolder)
.then(function() {
expect(this.elements.emailPlaceHolder.getAttribute("value")).to.equal('');
next();
});
}
*Thread Reply:* I’m not sure. Seems like a Selenium issue though. If you’re using the driver.wait function elsewhere I’d compare those calls to this one.
*Thread Reply:* I’ve used selenium in the past, and I recommend cypress because their docs are easier to read and better maintained.
*Thread Reply:* in some places i've noticed that findElement() sents an object with the promise instead of the webElement. Could it be an issue with the driver ?
*Thread Reply:* But in their case, they are declaring the findElement then awaiting the response. You are just having an issue with driver not being defined
*Thread Reply:* Thank you for your reply! I will try what you suggested
*Thread Reply:* @Joe A try { default: “--publish-quiet --format html:path/to/report.html” }
*Thread Reply:* Also see https://github.com/cucumber/cucumber-js/blob/main/docs/formatters.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* As explained in the doc, it really depends on your actual stack and needs.
What do you need to achieve? Which kind of server do you need to mock?
*Thread Reply:* I want to test the front end of an application, and I need to mock responses from the Backend. For example the case for reset password, I need to mock the token because otherwise it would be something random I couldn't implement some steps without knowing its value.
*Thread Reply:* I think I would implement a fake backend myself as this is fairly simple to build a http server using node-js.
Do you need to also spy the fake server? Or mocking it would be fine?
*Thread Reply:* if you're using nodejs for mocking the server do you need to run the server separatly ? Or can i mock the server in the in steps already implemented and run everything with only one command for the cucumber tests?
*Thread Reply:* You can basically do whatever you want actually
How do you execute your tests? Do you use npm scripts?
*Thread Reply:* I'm running them using this command : "node ./node_modules/selenium-cucumber-js/index.js -s ./step-definitions"
*Thread Reply:* I would recommend using npm scripts. To do so, in your package.json, add the following script:
"scripts": {
"test": "npm run test:cucumber",
"test:cucumber": "node ./node_modules/selenium-cucumber-js/index.js -s ./step-definitions"
}
Then to execute your tests, you would just have to run the following command: npm test.
Regarding your server, I would to managing it using hooks
You could start it using the beforeAll hook, and make sure to stop it in an afterAll hook.
You could also manage it using tags: you could have a tag which indicate that a scenario requires the server, then using Before and After hooks to start and stop the server per scenario depending their tags
*Thread Reply:* You may have another instance of Chrome driver installed
What is the result of which chromedriver ?
*Thread Reply:* Should it be installed in another way or with another command then?
*Thread Reply:* I've also tried to install chromedriver97 version and move it to usr/local/bin but it still says chromedrive not found
*Thread Reply:* Did you miss the final r or is it copy/paste issue?
*Thread Reply:* chromedriver -v still displays the wrong version number?
Make sure to also use the proper case for chromedriver. Don't know if it is chromeDriver or chromedriver
*Thread Reply:* i missed the r, the return is : /Users/manuelamag/.nvm/versions/node/v16.7.0/bin/chromedriver
*Thread Reply:* It seems you previously installed it globally (using npm i -g)
Try updating that one, or removing it to make sure using the local one
*Thread Reply:* after npm uninstall chromedriver it should use the local one? I tried but I got the same error
*Thread Reply:* no. Actually, npm uninstall chromedriver may have uninstalled the local one
Try to:
• npm uninstall -g chromecriver to uninstall your global chromedriver
• npm uninstall chromedriver to uninstall the local chromedriver if it is still there
• rm -rf node_modules to remove the node modules folder
• rm -f package-lock.json to remove the npm lock file
• npm install to reinstall all the packages
• npm install --save-dev chromedriver to reinstall chromedriver for your project
Disclaimer: uninstalling chromedriver globally may break other projects on your computer. If you want to keep the global one, make sure to update it npm update -g chromedriver but do not uninstall it. Using global packages is discouraged, but if you have some dependencies on it, you may keep it
*Thread Reply:* I tried the steps, but still shows the same error. I've removed the global driver
*Thread Reply:* Could you paste the "devDependencies" section of your package.json?
What is the result of which chromedriver ? chromedriver -v ? npx chromedriver -v?
Which error do you have? The following one?
This version of ChromeDriver only supports Chrome version 86
Current browser version is 97
*Thread Reply:* It seems that selenium-cucumber-jsactually depends on chromedriver v86: https://github.com/john-doherty/selenium-cucumber-js/blob/master/package.json#L43
If you want to still use that package, you may have to use chrome 86.
It seems selenium-cucumber-jsis not that maintained: https://github.com/john-doherty/selenium-cucumber-js/commits/master
You may consider to avoid using it and do the automation by yourself 😓
<a href="https://github.com/john-doherty/selenium-cucumber-js">john-doherty/selenium-cucumber-js</a>
*Thread Reply:* I see the problem now, thanks! Could you perhaps tell me what would be a better package or maybe an easier to follow example ? I'm new with cucumber-js.
*Thread Reply:* Well, maybe puppeteer with puppeteer-cucumber-js? https://github.com/john-doherty/selenium-cucumber-js
*Thread Reply:* I have another question, I see that puppeteer works with chromium, I search online but I did not find anything about the difference between chrome and chromium when it comes to testing. Do you know if there is a big difference between them?
*Thread Reply:* No difference between them for testing, don't worry 🙂
*Thread Reply:* There's certainly nothing built into Cucumber, but it's pretty straightforward to use a REST API. It looks like ServiceNow has one:
https://developer.servicenow.com/dev.do#!/reference/api/rome/rest/
*Thread Reply:* SuperAgent is a nice library for making REST calls: https://www.npmjs.com/package/superagent
*Thread Reply:* Or you can use node-fetch: https://www.npmjs.com/package/node-fetch
After @Josh Houlker asked for a typescript example the other day, I created a new repo of cucumber-js examples, here. It's a bit empty at the moment!
What are some other common setups you'd like to see examples for? A React app, for example, or Angular-JS? Using frameworks like Selenium, Cypress or Puppeteer? Please let me know!
*Thread Reply:* I get clients using React, Angular, Promises. I’ve managed to grok Promises enough to build a Jasmine example, long ago. But I don’t know enough about React or Angular to sling code with any authority. So, in brief: ALL OF THE ABOVE! 😂
*Thread Reply:* very nice. if I find the time I would like to add a 'fullstack' example with node backend and react frontend tested together in cucumber
*Thread Reply:* it would also be very helpful to have examples of mocking and assertions with libraries like chai or jest
*Thread Reply:* I’d really love to see examples of customizing the formatter. Something I’ve wanted to play around with in the past but don’t even know where to start
*Thread Reply:* hey @matt-SmartBear how do I contribute an example?
*Thread Reply:* 👋:skintone2: hi Matt.
Good question, we should probably document that!
I'm hoping you can clone the https://github.com/cucumber-examples/cucumber-js-examples repo, add a new folder in examples with your example, link to it from the README, then send us a pull request.
Does that make sense?
*Thread Reply:* makes perfect sense! but I had a permission denied error when I tried to push my branch ... what am I doing wrong? it should be from a fork?
*Thread Reply:* here is the example I've committed to a separate repo for now 😉 https://github.com/explosivose/cucumber-example-ts-node-react
*Thread Reply:* Did you push to a fork of the repo, or had you just cloned the one in the cucumber-examples org?
*Thread Reply:* If you try to push to the one in the cucumber-examples org, you'll get a permission denied as you're not a member of that org (yet)
*Thread Reply:* https://docs.github.com/en/get-started/quickstart/fork-a-repo
*Thread Reply:* That should let you push (to your fork) then you can make a pull request from there.
*Thread Reply:* Let me know if that doesn't make sense! Sorry there's not a contributor guide yet!
Dear CucumberJs people, at our company we are trying to setup a new test-system for our existing app using the setup CucumberJs, Typescript and Playwright. For our tests we would like to have access to the code of our own app (Angular + Typescript), for example to use some helper classes to do api calls.
Now, I have a hard time to import our app code in the tests. The problem is that we have some complited circular dependencies in our app code. For our app we use tsconfig.json -> module: es2020 with which we can manage the circular dependencies well. We use a index.ts file where we define the "correct" order to import the files. But, unfortunately CucumberJs does not support Typescript + es2020 Module, but needs us to use CommonJS (https://github.com/cucumber/cucumber-js/blob/main/docs/esm.md). But, I have a really hard time getting our circular dependencies to work using CommonJS as the CommonJs way of importing files is different.
I would appreciate any suggestion as I am stuck with this problem. Is there a chance that CucumberJs supports es2020 + Typescript in near future?
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Do you think it would be possible to share a public repo with a minimal reproducible example?
*Thread Reply:* Take a look at screenplay. https://github.com/cucumber/screenplay.js/
*Thread Reply:* I ran across it while looking at Playwright as a possible upgrade from Puppeteer which is what I'm currently using. I haven't had a chance to look at it in depth but it looks promising.
*Thread Reply:* Thank you for your responses. I decided to give up the attemp to integrate our app code into the cucumber tests because I encountared also other problems.
My original idea was to perhaps also write our unit/integration tests with cucumber where I would need direct access to the app code. Is this something you would recommend or would you say generally cucumber should be used only for end-to-end tests?
*Thread Reply:* I use Cucumber to test the javascript layer of a web application without working with it's actual API, which is quite complex. Using a concoction of Node, Express and Faker I generate test data, and I use Puppeteer's request interrupt feature to redirect API calls to this mock server.
That's why screenplay intrigues me if I understand it's readme correctly. It has the ability to abstract away this mocking from the main test so that it can run in both end2end and functional modes.
*Thread Reply:* ** https://cucumber.io/blog/the-worlds-most-misunderstood-collaboration-tool
*Thread Reply:* Not sure if you posted this link here or it was a bot, but found that article really good to better understand the intention of cucumber: https://cucumber.io/blog/the-worlds-most-misunderstood-collaboration-tool
*Thread Reply:* async/await is your friend if you aren't already using it
*Thread Reply:* The first scenario
``` Given('I land on Forgot password Page', async function () { await driver.get("https://dev.io/forgot-password"); }); When('I click on Submit', async function () { var submitButton = By.css('body > app-root > app-forgot-password > div > div > div > div:nth-child(2) > div.login.center > form > div:nth-child(2) > div > app-button > div > div > button'); var forgotEmail = By.css('body > app-root > app-forgot-password > div > div > div > div:nth-child(2) > div.login.center > form > div.form-group.mt-3.col-8.input-position > app-input > div > div > input');
await driver.findElement(forgotEmail).sendKeys("email"); await driver.wait(until.elementsLocated(submitButton), 10000); await driver.findElement(submitButton).click(); }); Then('I see the Success message', async function () { var error = By.css("#myModal > div > div > div > div.flex-item-content > div.modal-body"); var value = "An email has been sent to you with a link to reset your password."; await driver.wait(until.elementLocated(error, 10000)); driver.findElement(error) .then(function() { driver.findElement(error).getText() .then(function (state) { value = state; expect(value).to.equal(state); } ); });
// setTimeout(function() {
// }, 3000);
});
Second scenario:
Given('I land on Forgot password Page', async function () {
await driver.get("https://dev.io/forgot-password");
});
When('I click Back to login', async function () {
var backButton = By.css("body > app-root > app-forgot-password > div > div > div > div:nth-child(2) > div.login.center > form > p");
await driver.wait(until.elementLocated(backButton, 10000));
driver.findElement(backButton)
.then(function() {
driver.findElement(backButton).click();
}
);
await driver.getCurrentUrl()
.then( function(currentUrl){
expect("https://dev.io/login").to.equal(currentUrl);
});
});
Then('I am directed to the Login Page', async function () {
var backToLoginButton = By.css("body > app-root > app-forgot-password > div > div > div > div:nth-child(2) > div.login.center > form > p");
await driver.wait(until.elementsLocated(backToLoginButton), 10000);
await driver.findElement(backToLoginButton).click();
});```
*Thread Reply:* I recommend trying to simplify your selectors a bit. You should be able to remove some ">" and find descendants (with just a space) instead of direct descendants (here's a random resource: https://www.freecodecamp.org/news/css-selectors-cheat-sheet/). This will make your code cleaner.
It sounds like you are possibly running into some state issues. You could try: https://webdriver.io/docs/api/browser/reloadSession/ (if using webdriverIO)
Finally, this is not really a cucumber question, more of selenium/browser question, and you may find better support elsewhere (webdriverIO has a gitter chat, for example).
*Thread Reply:* I don't think you “tag as latest” in npm do you? That's a docker thing AFAIK
*Thread Reply:* It's unfortunate that when you install without specifying a version you get the RC. It'd be better if you got the latest stable instead.
*Thread Reply:* Oh I was wrong https://docs.npmjs.com/cli/v8/commands/npm-publish
*Thread Reply:* I’m not sure about npm and “latest”. I don’t publish a lot, sorry. I did think it was not supposed to pull in pre-releases.
*Thread Reply:* This default behavior has gotten me in the past as a publisher so I assumed it wasn't on purpose, but didn't want to jump to conclusions in a public forum! Thanks for weighing in David
*Thread Reply:* in the meantime i guess publishing a new patch version from the 7.x branch would make that latest and mitigate in the short term
*Thread Reply:* Could we use npm-dist-tag to tidy up? https://docs.npmjs.com/cli/v8/commands/npm-dist-tag
*Thread Reply:* Okay that’s done, so now our tags are:
latest: 7.3.2
next: 8.0.0-rc.2
(Given the convention of tagging betas and rcs as next in JS land)
*Thread Reply:* @matt-SmartBear watch out for a PR that updates our npm release action
*Thread Reply:* https://github.com/cucumber/action-publish-npm/pull/1
<a href="https://github.com/cucumber/action-publish-npm">cucumber/action-publish-npm</a>
*Thread Reply:* Nice one @David Goss! I've released v1.1.0 of the action-publish-npm GitHub action with this new input in it. How will we use it?
*Thread Reply:* just working that up in cucumber-js now, hold my beer
*Thread Reply:* behold https://github.com/cucumber/cucumber-js/pull/1905
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* if you're going from 6.x it might be smoother to go to 7.x first, then go from 7-8, especially since 8 is only a release candidate at the moment and may not be totally stable.
*Thread Reply:* Right now I’m trying to kind of hack it with { timeout: -1 } and wrapPromiseWithTimeout. A bit new to promises. Does someone know if I have to return for it to work? Not sure if I’m doing something else wrong.
*Thread Reply:* Also, sorry that I’m misusing it so badly. It’s [for a good cause, but still]
*Thread Reply:* If it’s not functioning, can we still use the global setDefaultTimeout()? I can experiment, I’m just hoping not to run into any gotcha’s later.
Either way, maybe the instructions for using it should be removed from the docs completely until it’s ready.
*Thread Reply:* looking at the changelog, wrapPromiseWithTimeout is from 8.0.0-rc1 - are you able to use that version? https://github.com/cucumber/cucumber-js/blob/main/CHANGELOG.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Hmm, I’m a little confused on a couple levels here, so I’ll try to articulate.
First, for my package: Since 8.0.0-rc.1 isn’t stable, I’m hesitant to use it for my package. Remembering to update it in the future may be complicated and easy to lose track of as well (esp. since I’m not convinced npm will update properly to the latest stable future version).
For others: Are the docs supposed to be for a non-stable version? Because right now the instructions for customizing a timeout has wrapPromiseWithTimeout.
For the migration docs: I may need to correct my update to the migration docs because they might be wrong at the moment. The v7 migration now says you can’t use setDefaultTimeout in a non-global context. That might not be true for v7. I’m not sure anymore.
*Thread Reply:* Is v8.0.0-rc.1 stable enough to count on for this stuff?
*Thread Reply:* that’s fair, i think it’s best to stay on stable for the reasons you say. in the meantime you could just grab the source of that function for now if its helpful https://github.com/cucumber/cucumber-js/blob/main/src/time.ts#L42
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* with the docs, yeah the docs in main can include stuff that isn’t released yet. https://github.com/cucumber/cucumber-js/tree/7.x is the right set for the current stable. this is definitely not ideal, we’ll review how to avoid it in future
*Thread Reply:* Our project is also struggling with how to handle multiple stable releases. We still haven’t figured out how to handle the docs for those. Badly, mostly.
*Thread Reply:* We’re thinking of removing the docs from the main branch and just referring to the ones in the specific version branches. At least, anything other than an overview.
*Thread Reply:* that does sound weird! May I ask why you're doing that?
*Thread Reply:* You certainly may! We’re misusing cucumber to write a testing framework for an open source platform that helps people create legal forms. Our team in particular is working on emergency legal forms (restraining orders, eviction stays, etc.), but we’re hoping to help the other users of the platform as well, most of who are pro bono legal services [and don’t have a lot of money or resources].
Most of these developers are actually not coders, [they’re lawyers]. We output reports to developers who run the tests to give them information that are easier for them to understand and have more relevant details. We want to be able to test the reports we output for them, even/especially when they’re describing errors. [That is, failing tests.]
I’m not sure that was a very clear explanation, so let me know if you have more questions.
*Thread Reply:* I'm intrigued, but still puzzled! Can you give me a real-world example?
*Thread Reply:* Very abstract example Scenario our users can write:
Scenario: I get a document
Given I start the interview at "important_form.yml"
When I tap to continue
And I set "user.name.first" to "Maria"
And I set "user.name.last" to "Gonzalez"
And I tap to continue
Then I download "landlord-notice.pdf"
There’s a more “story-driven” step, but it’d be harder to explain.
So, for our error catching, if puppeteer “tapped to continue” and hit an error screen, we’d print the text on the error screen in the report we make. We want to be able to test that our code adds the error text at appropriate times.
Is that the info you were looking for? Also, if you want a real-world example, I can point you to one, but it would be a lot to put here.
*Thread Reply:* @Manuela Mag that sounds like a question about Selenium WebDriver. In general it's better to use their forum for questions about that.
@Manuela Mag what assertion library are you using? You can use https://www.geeksforgeeks.org/node-js-assert-match-function/ to assert that a string matches a regex pattern. Or hamjest's matchesPattern https://github.com/rluba/hamjest/wiki/Matcher-documentation#matchespatternstringorregexp
https://github.com/purpleteam-labs/purpleteam-app-scanner/blob/main/src/api/app/models/app.js#L11
<a href="https://github.com/purpleteam-labs/purpleteam-app-scanner">purpleteam-labs/purpleteam-app-scanner</a>
*Thread Reply:* There is actually no documented list of the APIs.
I just meant that if you previously had used something from Cucumber that seems not available anymore because it is not explicitely exported, let us know
*Thread Reply:* I don’t think we have an official answer, but personally I’ve used and contributed to these: • https://github.com/vsiakka/gherkin-lint for linting feature files e.g. how many steps in a scenario, scenarios in a feature, correct use of And/But etc • https://github.com/darrinholst/eslint-plugin-cucumber for linting your JS step code e.g. no arrow functions, prefer one type of expression, etc
*Thread Reply:* You might try setting the driver on your World via a Before hook so that it’s available to each step at the start and you get a new one for each scenario that’s run https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/world.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* so to my understanding then I would add the POM to the world object and expose it as property?
*Thread Reply:* The only concern I have we going to have many POM objects that are exposed and going to be shared across 5 to 10 repos. So the world object may get big and messy quickly. I was hoping to get the POM initialize with the steps file is that still possible using the world object
*Thread Reply:* Yeah so you could have the driver live in the world but then initialise your page object in the step definition using the driver you grab from the world (this) - does that make sense?
*Thread Reply:* somewhat it does that mean i need to do it on each When or Then or just once at the step def?
*Thread Reply:* I was hoping once since not sure how many then and when will exists in each file
@binarymist if you have a suggestion for how we could better document the changes in the release notes for v8, that would be super helpful! It's sometimes hard for us to really get into our users' shoes and explain things from their perspetive.
*Thread Reply:* I’m tempted to disagree on this being a breaking change because it was never documented you could do it and arguably a quirk of CommonJS. But probably useful to call out since we know about it. Perhaps worth starting a new “ESM” heading here with a note about this https://github.com/cucumber/cucumber-js/blob/main/docs/migration.md - would you consider doing a PR for that @binarymist?
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Yip, happy to sub PR. I'll get everything working first along with some testing. How far away is 8.0.0?
*Thread Reply:* I don't see anything mentioned in docs about having to pass the env into instantiation of a Cli?
*Thread Reply:* This commit added the env into new Cli: https://github.com/cucumber/cucumber-js/commit/efff3a3516a6fd2f2a77de2f1dd6ac515349e8d6 @David Goss
*Thread Reply:* The CUCUMBER_PUBLISH_ENABLED doesn't appear to be new, but I don't see it mentioned in any doc other than issues, and now it seems it's required by the Cli
*Thread Reply:* It seems that getConfiguration has been remove from Cli somewhere between 7.2.1 and 8.0.0-rc.2. Is there some sort of replacement? Can you help with this @Aurélien Reeves?
*Thread Reply:* Hopefully you'll remember: https://github.com/purpleteam-labs/purpleteam-app-scanner_cucumber-1489-repro
*Thread Reply:* It looks like most of the methods have been removed from the Cli class. getConfiguration was removed in: https://github.com/cucumber/cucumber-js/commit/efff3a3516a6fd2f2a77de2f1dd6ac515349e8d6 and there doesn't appear to be a replacement @David Goss? Where should we find the same functionality?
*Thread Reply:* A lot happened actually as part of the following PR: https://github.com/cucumber/cucumber-js/pull/1849
You may consider taking some time to review it properly. That should help you with integrating with cucumber 8.0+
Actually before then, there were no documented API to execute cucumber programmatically. This is the purpose of the work done as part of v8: having a proper, documented way to interact with cucumber programmatically.
A lot may have changed, but actually you should be able to find the same things than before, they may have just moved.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* @David Goss I guess we may consider documenting that new API for the final 8.0.0?
*Thread Reply:* It's interesting that "no breaking changes" is mentioned a number of times in the PR you mentioned @Aurélien Reeves. I guess that part got forgotten in all the excitment?
> A lot may have changed, but actually you should be able to find the same things than before, they may have just moved. It appears as though getConfiguration has been removed rather than "just moved", at least I can't find it in the PR you mentioned. The implementation also appears to be removed rather than "just moved"
I'm really hoping I'm wrong..?..
*Thread Reply:* Can you show an example of how you're using the Cli class? I'm not sure why you'd need to call getConfiguration directly
*Thread Reply:* @binarymist actually what did change was not publicly documented APIs. From a public API point of view, the PR does actually add new things
When hacking on something using non-documented functionalities or internal APIs which are not mean to be used by third parties, you are subject to undocumented breaking changes. This is exactly what happens here.
*Thread Reply:* @David Goss As linked to above: https://github.com/purpleteam-labs/purpleteam-app-scanner/blob/main/src/api/app/models/app.js#L200 and the minimum repro example that @Aurélien Reeves helped with: https://github.com/purpleteam-labs/purpleteam-app-scanner_cucumber-1489-repro/blob/main/src/api/app/models/app.js
<a href="https://github.com/purpleteam-labs/purpleteam-app-scanner">purpleteam-labs/purpleteam-app-scanner</a>
<a href="https://github.com/purpleteam-labs/purpleteam-app-scanner_cucumber-1489-repro">purpleteam-labs/purpleteam-app-scanner_cucumber-1489-repro</a>
*Thread Reply:* OK @Aurélien Reeves, so now that the API has been removed, what should we use instead?
*Thread Reply:* The new one issued from the PR#1849
If something is missing for you, you can take part of the discussion here: https://github.com/cucumber/cucumber-js/issues/1711 or in the PR
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* The new one appears to be missing the functionality @Aurélien Reeves, does it not? Thanks for the link.
*Thread Reply:* I think you're looking for <#C5YHPPJMP|help-cucumber-jvm> - you should be able to get some support in there
*Thread Reply:* which version of cucumber are you using?
latest version may have some improvements regarding windows support
Also what "does not work" mean exactly? Do you have more info? Some error message maybe?
*Thread Reply:* Thanks.
Take a look at the following, there is a workaround for Windows: https://github.com/cucumber/cucumber-js/issues/1796#issuecomment-938405981
The issue is actually related to npm
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Could you give more info about what you are trying to achieve?
You want to update ENV as part of your run? To set some kind of default values, or forcing some specific values?
*Thread Reply:* i want to pass env variable directly to the process.env from the cucumber.js or something similar
*Thread Reply:* Why do you need to do that as part of the cucumber run? Setting appropriate ENV before executing the run is not an option?
Did you tried the BeforeAll hook? (https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/hooks.md#beforeall--afterall)
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* i wanted it to be as part of the settings and then it can be env changeable but if there is no option i will use the before all
*Thread Reply:* I believe you need to use the --require-module ts-node/register flag. you could also add it to your test-cukes script i believe
*Thread Reply:* Hey @Robert.S, while as a general rule I would advise leaning into the defaults where possible, you can indeed have your features in whatever directory you like, you just need to tell cucumber by passing one or more globs as the positional arguments. So for your example say you are currently running
cucumber-js <your options>
You’d need to change to
cucumber-js src/cucumber-js-example/features/****/**.feature <your options>
Full docs are here https://github.com/cucumber/cucumber-js/blob/main/docs/cli.md.
*Thread Reply:* @David Goss, thanks for your help and pointing me to the documentation.
It would appear my earlier problem was a result of placing the path to the feature files in the wrong location within the cucumber.js file.
Here is what I had
module.exports = {
default: [
'src/cucumber-js-example/features/****/**.feature',
'--require-module ts-node/register',
'--require src/cucumber-js-example/features/step_defiinitions/****/**.ts',
'--publish-quiet',
'--format @cucumber/pretty-formatter',
].join(' '),
};
*Thread Reply:* And, after you guidance, I updated it to be
module.exports = {
default: [
'--require-module ts-node/register',
'--require src/cucumber-js-example/features/step_defiinitions/****/**.ts',
'--publish-quiet',
'--format @cucumber/pretty-formatter',
'src/cucumber-js-example/features/****/**.feature',
].join(' '),
};
And, it worked 😄 !!!. The path to features has be the last item in the list
*Thread Reply:* note that there are a few examples to refer to here: https://github.com/cucumber-examples/cucumber-js-examples
*Thread Reply:* thanks @matt-SmartBear. That’s the first resource I looked at. Unfortunately, the documentation does not mention anything about required placement of the ‘features’ path within the cucumber.js file
*Thread Reply:* @Robert.S Will it because of the typo in step_defiinitions?
*Thread Reply:* My goal is to only run certain cleanup task in AfterAll if a certain tag is applied. I don't see any way to do that right now.
*Thread Reply:* I'm surprised. The --require should require all JavaScript files in that folder at runtime.
*Thread Reply:* Can you try and create a minimal reproducible example on GitHub?
*Thread Reply:* I would be more surprised if requireing an executable like that worked, they aren’t designed for that. What’s stopping you from providing --require with the paths to the world and steps files?
*Thread Reply:* (I think Cucumber is probably treating ./node_modules/.bin/alkiln-cucumber as a directory not a file path, because it has no extension.)
*Thread Reply:* These lines to run executables are working:
"setup": "node_modules/.bin/alkiln-setup",
"takedown": "node_modules/.bin/alkiln-takedown"
That’s from the dependent’s package.json. Those lead to files, not directories. In our package that looks like:
"bin": {
"alkiln-setup": "./lib/docassemble/setup.js",
"alkiln-takedown": "./lib/docassemble/takedown.js",
"alkiln-cucumber": "./lib/index.js",
}
*Thread Reply:* Small reproducible example: https://github.com/plocket/exec-dependent
*Thread Reply:* Thanks for the repro, added a PR here https://github.com/plocket/exec-dependent/pull/1
<a href="https://github.com/plocket/exec-dependent">plocket/exec-dependent</a>
*Thread Reply:* Basically anything you pass to --require with no extension will be treated as a directory, not a file path, so you need to give it a path to the actual file(s) you want, not the executable script
*Thread Reply:* Sorry that I didn’t actually repro the right issue and thanks for finding out about --require. Good thing to know in general 👍
*Thread Reply:* No problem! I think issues with configuration like this are too hard to diagnose at the moment, so I'm looking at adding a debug option to make it easier.
Hey all! I'm trying to use cucumber-js 8.0.0-rc.2 with ES modules, and in the docs here: https://github.com/cucumber/cucumber-js/blob/main/docs/cli.md I see there there is an --import option, but when I try to use it, i get the following error:
error: unknown option '--import'
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I'm guessing this feature hasn't been released yet? Any ideas when I can start using it?
*Thread Reply:* Hey @Mark W. Oosterveld, rc3 should be out in the next few days with that available
*Thread Reply:* I'm starting a new (hobby) project, and I plan to use ESM and Typescript
*Thread Reply:* Not yet but I’ll be looking at that next. Oddly enough just commented here https://github.com/cucumber/cucumber-js/issues/1844#issuecomment-1066174669
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I don’t think we have a first-class way to support this now, but you might be able to work around it, something like:
```// runs once before everything BeforeAll(() => { process.env.BAILONTEST_RUN = 'true' })
// runs before each scenario and can skip it Before(() => { if (process.env.BAILONTEST_RUN === 'true') { return 'skipped' } })```
*Thread Reply:* This should be a bit less clunky once https://github.com/cucumber/cucumber-js/pull/1770 is finished and we have a global test run context.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Personally I like this package, it allows you to set a property in your .npmrc file that always installs the latest version of chromedriver https://www.npmjs.com/package/chromedriver
*Thread Reply:* Thank you! Could you also tell me which property is that? And how to find that file? I'm using a macOs big sur
*Thread Reply:* No problem! The property is chromedriver_version=LATEST, you would have to create a .npmrc file in your project (that is, assuming your project is a NodeJS one?): https://docs.npmjs.com/cli/v8/configuring-npm/npmrc
*Thread Reply:* It’s worth noting that this package will install chromedriver in your project’s node_modules folder, I personally like that but some people prefer it to be installed globally somewhere
*Thread Reply:* i think it's quite useful to have this file with that property, so I would not have to change manually the chrome version each time it becomes deprecated
*Thread Reply:* What else should the file contain? I'm searching right now some example, but i didn't find a good one yet
*Thread Reply:* It doesn’t need to contain anything else unless you want it to, it only exists to pass config to npm packages when you’re installing, so the file can just have the above mentioned property in it (unless you have other properties you want to override)
*Thread Reply:* Hmm I used to use the latest flag I think, but I reverted to changing versions manually because chromedriver always updates before the latest version of chrome is available for download. 😞
*Thread Reply:* There’s always going to be the potential for the browser and driver versions to get out of sync, the only way I’ve been able to consistently keep them in line is to install Chromium as a dependency in my project as well, and make sure that the version of Chromium matches the version of Chromedriver. Then I can bump them to newer versions at the same time when I choose to do so
*Thread Reply:* Just fyi, that last one (with password) has a typo or two. I think Joe A is right in general, though - it looks like the code isn’t finding that file. Where do you require the different files? And what does your package.json scripts look like?
*Thread Reply:* The file is spelled incorrectly for cucumber to find it without a setting. the image says step-definAtion but cucumber will only look for step_definItion unless you tell it otherwise (I believe)
*Thread Reply:* here is the code snippet: ```When(
'{pronoun} select a country code and provide a valid mobile number',
async (actor: Actor, table: DataTable) => {
const countryCode = table.hashes()[0].countryCode;
const mobileNumber = table.hashes()[0].mobileNumber;
await LoginPage.selectCountryCode();
await LoginPage.enterMobileNumber(mobileNumber);
await LoginPage.continueToOTP();
await LoginPage.waitForOTPToProcess();
},
);
Then('{pronoun} is able to verify using OTP', async (actor: Actor) => {
// need to get the value countryCode and mobileNumber from the When step
const date = new Date();
const timeStamp = date.toUTCString();
<a href="http://console.info">console.info</a>('timestamp : ' + timeStamp + '\ncountryCode: ' + countryCode + '\nmobileNumber: ' + mobileNumber)
const OTP = await LoginPage.readOTP(timeStamp, countryCode + mobileNumber);
<a href="http://console.info">console.info</a>('OTP Number : ' + OTP);
await LoginPage.enterOTPInput(OTP);
});```
*Thread Reply:* This is tipically the job of the world: https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/world.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I’m not aware of a way to do this, might it be easier to increase the default size of the viewport by the height of the info bar?
*Thread Reply:* I think this is an issue with chrome web driver, not cucumber.
*Thread Reply:* You might have better luck asking on their github or if they have a slack
*Thread Reply:* @Rabin There is a built in support for cucumber in nightwatch itself maybe you can use that
*Thread Reply:* Hard to know without more details but the new functionality (just released in 8.0.0) around custom work assignment might help you https://github.com/cucumber/cucumber-js/blob/main/docs/parallel.md#custom-work-assignment - you might be able to allow parallel execution generally whilst limiting your constrained-user scenarios to one at a time
Hi all, I have been using json formatter in cucumber and sometimes the it throws a type error inside cucumber here: https://github.com/cucumber/cucumber-js/blob/main/src/formatter/json_formatter.ts#L278 it says testStepResult is undefined
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Haven’t tried to do anything like this yet and someone will probably have better advice. That being said, this sounds like an afterEach hook deal. Creating some function that writes a file. I’m not sure if afterEach hooks will run if you’re doing a scenario outline, or if the tests need to be separate.
*Thread Reply:* Thanks for the reply. I already after that function in place in my After(scenario) hook. I am able to write a file after each scenario completes and I can even append a test case ID generated by cucumber-js using something like:
After((scenario) => {
const filename = `${scenario.pickle.name}-${scenario.testCaseStartedId}`;
fs.writeFile(`./tests-output/${filename}.json`, outputData, (err) => {
if (err) {
throw new Error(err);
}
});
*Thread Reply:* I got the name wrong, it’s AfterStep. https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/hooks.md#beforestep--afterstep
*Thread Reply:* But if you already had something like that in place, you could add state to your cucumber world to keep track of the “id”.
example:
this.counter = 0;
fs.fileWrite(`./tests-output/${filename}_${counter}.json`);
this.counter++;
*Thread Reply:* Yeah, unfortunately, adding a counter doesn’t tell me if the file created is the result of a retry or an example in Scenario Outline.
*Thread Reply:* Is there info about whether the test failed or passed in the after step?
*Thread Reply:* Yes, I can get the following when doing a console.log(result):
*Thread Reply:* {
duration: { seconds: 1, nanos: 971000000 },
status: 'PASSED',
message: undefined,
willBeRetried: false
}
*Thread Reply:* You can use state to manage that as well.
this.isRetry = status === failed;
if (this.isRetry) {
fs.fileWrite(`./tests-output/${filename}_${retry}_${counter}.json`);
}
*Thread Reply:* obviously that code won’t work directly but you get the idea.
*Thread Reply:* I can do something like that. You’re on the right track (we just need to handle the case where status === FAILED but there is no more retry after that i.e. willBeRetried === false ). I was just hoping that cucumber-js offered that out of the box.
*Thread Reply:* That works for the retries. I can implement a similar mechanism for examples in a Scenario Outline. I think we can conclude that a scenario is an example if the scenario name is the same.
*Thread Reply:* Right now the best workaround is probably to template your feature file and have a pre test step that generates it with your data inlined.
*Thread Reply:* There is a proposal here if you want to weigh in https://github.com/cucumber/common/issues/773
<a href="https://github.com/cucumber/common">cucumber/common</a>
*Thread Reply:* Thanks @David Goss. That’s helpful information.
Right now the best workaround is probably to template your feature file and have a pre test step that generates it with your data inlined.
I think that means setting the data into context for the scenario to use.
*Thread Reply:* No, what I meant was having your feature file exist in your repo as a template (e.g. handlebars or mustache), and having a little script or build step that uses the template to generate a feature file with an examples data from all the data, then running cucumber after that.
*Thread Reply:* ahh, I get what you mean now. Yes, all good, I think there are multiple options. But there is no built-in solution which is what I initially asked for.
*Thread Reply:* I ended up by creating npm package for it, and publishing it to the npm private registy.
*Thread Reply:* But I think we can fix before 8.0.0. I don’t think we anticipated that custom formatters might be written in TS and effectively benefit from ts-node being require’d via --require-module earlier in the process. We should be able to rework to accomodate
*Thread Reply:* here we go https://github.com/cucumber/cucumber-js/pull/1985
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* @David Goss Thanks for the quick fix! Will check later today
*Thread Reply:* You might find using the new runCucumber API helps with this, since you can load the support code once and use it on multiple runs in the same process
https://github.com/cucumber/cucumber-js/blob/main/docs/javascript_api.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
📢 cucumber-js 8.0.0 is out https://cucumber.io/blog/open-source/announcing-cucumber-js-8-0-0/ - please give it a try and feed back if you can!
*Thread Reply:* You can configure cucumber.js to require your ts files
*Thread Reply:* I have the following for a repo using ts:
const common = [
"--require-module ts-node/register",
"--require test.setup.ts", // Load hooks
"--require features/stepDefinitions/****/**.ts", // Load step definitions
"--format progress", // Load custom formatter
].join(" ");
*Thread Reply:* @Hatch yes that would be in your cucumber.js config file
*Thread Reply:* you can just pass those options on the CLI if you want to try it out, but it's easier to have them save off as default
*Thread Reply:* Sorry I missed the follow-up question; I could have been more explicit.
*Thread Reply:* I should note that I've still got some projects using Cucumber 7 and that upload step passes
*Thread Reply:* I see from the CHANGELOG (https://github.com/cucumber/cucumber-js/blob/main/CHANGELOG.md#changed-4) that this was included in the v8 release: https://github.com/cucumber/cucumber-js/pull/1793
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Aha, well spotted! Looks like the number of decimal places should have been capped as part of that work, I wonder if something's regressed since the feedback was addressed?
*Thread Reply:* It seems you are using a tool written in Go to process the json. Is it one issued from cucumber? Or something else?
*Thread Reply:* It’s AWS CodeBuild’s built in test report processor, which I can only assume is written in Go: https://docs.aws.amazon.com/codebuild/latest/userguide/test-reporting.html
*Thread Reply:* Ok, thanks
I guess we may submit an issue there, or even a PR, to support the new duration precision of cucumber-js
*Thread Reply:* I though that was a link to a github repo
Trying to find one. Not sure this is actually open-source
*Thread Reply:* Just submit an issue here: https://github.com/awsdocs/aws-codebuild-user-guide/issues/91
<a href="https://github.com/awsdocs/aws-codebuild-user-guide">awsdocs/aws-codebuild-user-guide</a>
*Thread Reply:* Awesome, thank you! Hopefully that will get the ball rolling on a fix. In the meantime I’ve had to downgrade to 7.3.2, which is a shame
*Thread Reply:* test:execute:android] [0-0] at tokenize (/Users/a845751yara.com/dev/sh-usermanagement-native-poc/nodemodules/@wdio/cucumber-framework/nodemodules/@cucumber/tag-expressions/src/index.ts:86:20) [test:execute:android] [0-0] at Object.parse [as default] (/Users/a845751yara.com/dev/sh-usermanagement-native-poc/nodemodules/@wdio/cucumber-framework/nodemodules/@cucumber/tag-expressions/src/index.ts:23:18) [test:execute:android] [0-0] at new PickleTagFilter (/Users/a845751yara.com/dev/sh-usermanagement-native-poc/nodemodules/@wdio/cucumber-framework/nodemodules/@cucumber/cucumber/src/picklefilter.ts:128:37) [test:execute:android] [0-0] at new PickleFilter (/Users/a845751yara.com/dev/sh-usermanagement-native-poc/nodemodules/@wdio/cucumber-framework/nodemodules/@cucumber/cucumber/src/picklefilter.ts:36:22) [test:execute:android] [0-0] at new CucumberAdapter (/Users/a845751yara.com/dev/sh-usermanagement-native-poc/nodemodules/@wdio/cucumber-framework/build/index.js:79:30) [test:execute:android] [0-0] at Object.adapterFactory.init (/Users/a845751yara.com/dev/sh-usermanagement-native-poc/nodemodules/@wdio/cucumber-framework/build/index.js:319:21) [test:execute:android] [0-0] at Runner.run (/Users/a845751yara.com/dev/sh-usermanagement-native-poc/nodemodules/@wdio/runner/build/index.js:83:49) [test:execute:android] [0-0] at processTicksAndRejections (node:internal/process/taskqueues:96:5) [te
*Thread Reply:* here is the complete set of cucumber opts in the wdio config file
*Thread Reply:* // If you are using Cucumber you need to specify the location of your step definitions.
cucumberOpts: {
// <string[]> (file/dir) require files before executing features
require: ['./tests/src/****/**.ts'],
// <boolean> show full backtrace for errors
backtrace: false,
// <string[]> ("extension:module") require files with the given EXTENSION after requiring MODULE (repeatable)
requireModule: [],
// <boolean> invoke formatters without executing steps
dryRun: false,
// <boolean> abort the run on first failure
failFast: false,
// <string[]> (type[:path]) specify the output format, optionally supply PATH to redirect formatter output (repeatable)
format: ['pretty'],
// <boolean> hide step definition snippets for pending steps
snippets: true,
// <boolean> hide source uris
source: true,
// <string[]> (name) specify the profile to use
profile: [],
// <boolean> fail if there are any undefined or pending steps
strict: false,
// <string> (expression) only execute the features or scenarios with tags matching the expression
tagExpression: ['@universal', '@embedded'],
// <number> timeout for step definitions
timeout: 1000000,
// <boolean> Enable this config to treat undefined definitions as warnings.
ignoreUndefinedDefinitions: false,
},
*Thread Reply:* I’m not sure what mapping might be happening with wdio, but for Cucumber the tags config option is a single string, not an array.
You might want something like '@universal and @embedded'
*Thread Reply:* Docs; • https://github.com/cucumber/cucumber-js/blob/main/docs/filtering.md#tags • https://cucumber.io/docs/cucumber/api/#tag-expressions Hope this helps!
*Thread Reply:* (I just fixed a typo in that first doc that made it slightly misleading - apologies if that’s what you were going off!)
*Thread Reply:* it didn’t work. i can only use tagExpression = ‘@embedded’
*Thread Reply:* https://github.com/badeball/cypress-cucumber-preprocessor
*Thread Reply:* I thought that library was deprecated hahaha https://github.com/badeball/cypress-cucumber-preprocessor-deprecated yesterday I saw the URL without "-deprecated" but it said deprecated in its content or my bad, i dont know 😅
Sorry to come here with this question, but I’ve seen a lot of people in here talking about puppeteerjs and I’m not sure where else to go. I’m using puppeteer’s page.on('response'…) listener, but I can’t see any of the console logs I make in there. Does anyone know how I can see those logs? I’m trying to rename a file I download and it’s not working at all - nothing’s happening to the name whatsoever. I’m trying to log to troubleshoot, but I don’t see any logs in the command prompt or in the browser running in headless mode. Here’s the code before I’ve added the logs: https://github.com/SuffolkLITLab/ALKiln/blob/552_artifacts_dirs/lib/steps.js#L214. Strangely enough, I do think the Step does wait for the doc to finish downloading because the process waits the appropriate amount of time before exiting.
<a href="https://github.com/SuffolkLITLab/ALKiln">SuffolkLITLab/ALKiln</a>
*Thread Reply:* Not that I know of. @Aslak Hellesøy would this ever be in scope of our VSCode extension in future?
*Thread Reply:* @Eliav Ran @David Goss there is an issue you can monitor here: https://github.com/cucumber/vscode/issues/69
<a href="https://github.com/cucumber/vscode">cucumber/vscode</a>
*Thread Reply:* Hey, you might want to check the example in this issue https://github.com/cucumber/cucumber-js/issues/1851 which I think might be what you need.
Also, would be great if you could comment there with your use case - we’ll definitely do something to address this, just want to make sure it solves the right problem.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Yep - much more neutral advice there. The Cucumber position is essentially that a global namespace forces you to be deliberate and specific about your terminology, and helps highlight when things get conflated and misused. I think @matt-SmartBear had some good quotes.
*Thread Reply:* There is some more context here: https://github.com/cucumber/common/issues/1949
<a href="https://github.com/cucumber/common">cucumber/common</a>
*Thread Reply:* Ha 🤔interesting discussion/view on things. I might indeed start out to let the team write more feature specific gherkin, and avoid hacky generic language. Thnx for the link !
*Thread Reply:* hmm - it shouldnt include the ascii color stuff if you direct it to a file. which version of cucumber-js?
*Thread Reply:* Like build mine own report by using messages? I just don't quite understand how messages works for this.
*Thread Reply:* The docs https://github.com/cucumber/common/tree/main/messages is not built for that purpose I think.
*Thread Reply:* try the latest (8.2.2) - we put some better handling in for when (not) to emit colors based on the output stream and other stuff
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* (sending the summary formatter to a file seems a good low effort way to do this. but yeah you could do a custom formatter also)
*Thread Reply:* I just couldn't find a straightforward tutorial/example for how to use message to build my custom report.
*Thread Reply:* I guess some API changed in V8? I now get error Error: Cannot find module '/node_modules/@cucumber/cucumber/lib/formatter/helpers.js'
*Thread Reply:* cool. I can see the special chars & color code are gone.
*Thread Reply:* parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 9, column 24
still fail on slack
*Thread Reply:* I think it's due to some jq and JSON format issue in the Slack integration. I end up writing my own from parsing the old JSON formatter.
*Thread Reply:* When I console log in various places in the Before() callback, all the logs get logged.
*Thread Reply:* This package has been working for a while and this is a new bug. I’m not sure how to track it down because there’s so little information. It’s a pretty big package. Is there more I can tell you?
*Thread Reply:* Hmm, Slack changed. Can’t find how to add a code snippet, so this’ll be a long code block…
*Thread Reply:* ```git diff diff --cc .github/workflows/testtests.yml index 8643da4,97f6f16..0000000 --- a/.github/workflows/testtests.yml +++ b/.github/workflows/test_tests.yml diff --cc action.yml index aadaccc,c700978..0000000 --- a/action.yml +++ b/action.yml diff --cc lib/scope.js index 61a15db,e409330..0000000 --- a/lib/scope.js +++ b/lib/scope.js @@@ -60,21 -63,7 +63,25 @@@ module.exports = let scenario = await scope.makeSureReportPartsExist( scope ); scenario.lines.push({ type, value });
- log.addtodebug_log(
Report:\ntype: ${ type }, value: ${ value }\n); ++// <<<<<<< HEAD - // Ensure filepath for saving local unit test logs
- if ( !scope.paths ) {
- // TODO: would it be useful to put in existing artifacts folder if possible. Can this be usefully
- // abstracted?
- let date = await scope.dateforfilename();
- scope.paths = { debug_log:
_alkiln_test-misc_artifacts/unit_tests_logs-${ date }.txt}; - }
+
- try {
- fs.appendFileSync(scope.paths.debuglog,
\nReport:\ntype: ${ type }, value: ${ value }); - } catch (err) {
- fs.mkdirSync(
_alkiln_test-misc_artifacts); - fs.appendFileSync(scope.paths.debuglog,
\nReport:\ntype: ${ type }, value: ${ value }); - }
++// try {
++// fs.appendFileSync(,
\nReport:\ntype: ${ type }, value: ${ value }); ++// } catch (err) { ++// fs.mkdirSync(_alkiln_test-misc_artifacts); ++// fs.appendFileSync(scope.paths.debug_log,\nReport:\ntype: ${ type }, value: ${ value }); ++// }
++// ======= ++ log.addtodebuglog(
Report:\ntype: ${ type }, value: ${ value }\n, scope.paths.debuglog ); ++// >>>>>>> releases/v4 return scenario;}, // Ends scope.addToReport() @@@ -504,6 -478,44 +511,45 @@@ await scope.tapElement( scope, elem, tab_id ); },
- checkForA11y: async function (scope ) {
- /** Return if there were any violations and the file containing aXe output with more details of the checks **/
- let { id } = await scope.examinePageID( scope, 'none to match' );
- if ( scope.page_id !== id ) {
- await scope.addToReport(scope, { type:
page_id, value:${ id }, }); - scope.page_id = id;
- }
- const axe = new AxePuppeteer(scope.page);
- axe.options({
- runOnly: ["wcag2a", "wcag2aa", "wcag21a", "wcag21aa"],
- resultTypes: ['violations', 'incomplete']
- });
- let axe_result = await axe.analyze();
- axeresult.pageid = scope.page_id;
- axeresult.passesids = axeresult.passes.map(passobj => pass_obj.id);
- axeresult.inapplicableids = axeresult.inapplicable.map(inappobj => inapp_obj.id);
- axe_result.passes = [];
- axe_result.inapplicable = [];
-
- let axe_filename = null;
- const passed = axe_result.violations.length == 0;
- if (!passed) { ++ // TODO: Add timestamp to filename
- axe_filename = await scope.getSafeScenarioFilename( scope, {prefix:
aXe_failure_${ scope.page_id }}) + '.json';
fs.writeFileSync( axefilename, JSON.stringify( axeresult, null, 2 )); ++ fs.writeFileSync(
${ scope.paths.scenario }/${ axe_filename }, JSON.stringify( axe_result, null, 2 ));- let msg =
Found potential accessibility issues on the ${ scope.page_id } screen. Details in ${ axe_filename }.; - await scope.addToReport( scope, {
- type:
error, - value: msg
- });
- } else {
- await scope.addToReport(scope, { type:
info, value:Accessibility on ${ scope.page_id } passed!}) - }
- return {
- passed: axe_result.violations.length == 0,
- axefilename: axefilename
- };
- },
tapElementBySelector: async function ( scope, selector, waitForTimeout ) { let elem = await scope.page.$(
${selector}); let error_msg =Couldn't find "${selector}" on the page, is there a typo?; diff --cc lib/steps.js index 3a9a326,c59032e..0000000 --- a/lib/steps.js +++ b/lib/steps.js @@@ -18,9 -18,8 +18,10 @@@ const { v4: uuidv4 } = require('uuid')// Ours const scope = require('./scope');
const log = require('./utils/log'); const sessionvars = require('./utils/sessionvars'); +const makeartifactsfolder = require('./utils/makeartifactsfolder' ); + /** Of Note:
We're using
**=because sometimes da text has funny characters in it that are hard to anticipate **/ @@@ -92,13 -77,9 +94,15 @@@ Before(async (scenario) => scope.scenarioid = uuidv4(); scope.expectedstatus = null; scope.expectedinreport = null; ++ console.log(1);await scope.addReportHeading(scope, {scenario}); scope.base_filename = await scope.getSafeScenarioBaseFilename(scope, {scenario});
- let msg =
- let date = await scope.dateforfilename();
- scope.paths.scenario =
${ scope.paths.artifacts }/${ scope.base_filename }-${ date }; - scope.paths.scenario_report =
${ scope.paths.scenario }/report.txt; fs.mkdirSync( scope.paths.scenario ); ++ console.log(2);
// Downloads scope.downloadComplete = false; @@@ -107,8 -88,10 +111,12 @@@ scope.device = 'pc'; scope.activate = clickwith[ scope.device ]; scope.disableerror_screenshot = false; ++ console.log(3);
scope.page_id = null;
- scope.checkallfor_a11y = false;
scope.passedallfor_a11y = true; ++ console.log(4);
// Reset default timeout scope.timeout = 30 ** 1000; diff --cc tests/features/observationsteps.feature index 5c5646f,d09cd3f..0000000 --- a/tests/features/observationsteps.feature +++ b/tests/features/observationsteps.feature @@@ -123,7 -123,7 +123,7 @@@ Scenario: I get the page's JSO Given I start the interview at "alltests" Then I get the page's JSON variables and values
@fast @o9 @screenshot -@fast @o11 ++@fast @o11 @screenshot Scenario: I take a screenshot Given I start the interview at "all_tests" Then I take a screenshot diff --git a/lib/utils/log.js b/lib/utils/log.js index ee9960e..3ec329c 100644 --- a/lib/utils/log.js +++ b/lib/utils/log.js @@ -51,8 +51,24 @@ log.debug = function ({ type='', pre='', data='', post='' }) { } };
-log.addtodebuglog = function(value) {
- fs.appendFileSync(log.debuglogfile, value);
+log.addtodebuglog = function(value, pathname) {
+ /* Try to add value to file at optional path. On failure, try to add to file at a default
+ * directory. If that doesn't exist, make that default directory and then put log there.
+ *
+ * This complexity is mostly for unit tests, which don't otherwise go through the artifact
+ * folder creation process. */
+ try {
+ fs.appendFileSync(pathname, value);
+ } catch (err) {
+ try {
+ fs.appendFileSync(${ log.fallback_debug_dir }/${ log.debug_log_file }, value);
+ } catch (err) {
+ console.log( err );
+ fs.mkdirSync( log.fallbackdebugdir );
+ fs.appendFileSync(${ log.fallback_debug_dir }/${ log.debug_log_file }, value);
+ }
+ }
};
log.debuglogfile = 'debuglog.txt'
+log.fallbackdebug_dir = _alkiln_test-misc_artifacts;```
*Thread Reply:* I can commit it to a different branch and push it there. That may take a bit.
*Thread Reply:* I’m just going to look at what changed from when it was working to now and debug it that way.
*Thread Reply:* This is the branch: https://github.com/SuffolkLITLab/ALKiln/tree/552_artifacts_dirs
*Thread Reply:* This is the branch I’m merging into it: https://github.com/SuffolkLITLab/ALKiln/tree/releases/v4
*Thread Reply:* Can you open a pull request between the two branches, so I can see the diff?
*Thread Reply:* https://github.com/SuffolkLITLab/ALKiln/pull/567
<a href="https://github.com/SuffolkLITLab/ALKiln">SuffolkLITLab/ALKiln</a>
*Thread Reply:* Nothing super obvious jumps out. there’s a lot going on in the before, hard to tell what could possibly be causing it
*Thread Reply:* Or flag a specific Scenario for retesting [while that scenario is being run]?
*Thread Reply:* Not at the moment. Only tags can influence whether retry kicks in.
*Thread Reply:* Hi Tom,
This is a known thing. It has been discussed a few weeks ago here: https://cucumberbdd.slack.com/archives/C6QJ6N695/p1651483291178619
As you can see, I have submitted an issue to AWS, but it seems they do not care about it 😢
From our side, right now we have no plan to revert our changes
However if someone submit a PR to add an option that would disable the performance counter on demand, I guess that would be accepted
*Thread Reply:* do you think you could open a feature request suggesting that here: github.com/cucumber/cucumber-js/issues?
*Thread Reply:* Hey Aurélien, thank you for the quick reply.
To confirm, you want me to create a feature request on the cucumber github to add a config to disable the duration counter?
I'm happy to do that I just want to be sure 😄
*Thread Reply:* https://github.com/cucumber/cucumber-js/issues/2086
Please see this and let me know if it's satisfactory
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I forgot to mention, I actually raised this directly with AWS support a few days ago after not hearing anything on the GitHub issue. This was the reply:
```Hello,
Thank you for contacting AWS Premium Support. My name is Kieran and I will be assisting you today!
I understand on upgrading Cucumber-js to version 8.0, CodeBuild began failing with the following message:
—— Error in UPLOAD_ARTIFACTS phase: [system: [error processing report: [/codebuild/output/src381885655/src/reports/system/cucumber.json: json: cannot unmarshal number 253118.99999999997 into Go struct field Result.elements.steps.result.duration of type time.Duration]]] ——
I have researched the 'cannot unmarshal number' message on my side and can see that this is a known Golang issue which you can read more about here [1]. The error here describes the JSON duration being unable to be marshalled to the 'time.Duration' type.
As this is a Golang issue and not a CodeBuild issue, unfortunately, this falls outside of AWS' Scope of Support [2].
However, I have looked further into the issue. Please see the blog post [3] discussing marshalling methods for Go. It includes example code for handling of numeric or string representations.
I hope our chat today helped. If you have further concerns or queries, please do not hesitate to reach out. I am always happy to help!
Have a great day!``` I’m going to push back, as it’s their json parser that has the issue
*Thread Reply:* @Jim Trenowden do you have a link or case # for that case you raised with AWS? I might have a way to get it in front of the right people.
*Thread Reply:* Hi @David Goss did you have any luck with escalating this to the right people at AWS?
*Thread Reply:* Yes, somewhat. For now they’re going to call out 7.x as their highest supported cucumber-js version, and a fix for the unmarshalling error is on their backlog - no timescale though
*Thread Reply:* Ace, thank you. Is the plan currently to implement the opt-out option you spoke about before or wait for AWS to address the error on their end?
*Thread Reply:* To be honest I’m minded to just change the JSON formatter to Math.floor those timestamps rather than make an option. I think this was a regression/oversight on our part (core team) that we should correct. Hopefully AWS will also make their code more flexible and then the affected versions will work for them as well.
*Thread Reply:* That's awesome, do you anticipate a timeframe for this? Appreciate there's a lot you folks do just so I can take it back to my boss 😄
*Thread Reply:* Salutations Earthling! "Guys" is easily replaced with the more gender-neutral "Folks" or "All". We'd appreciate if you tried to use that. Thanks!
*Thread Reply:* https://github.com/cucumber/cucumber-js/pull/2094 - hoping to get this in and released in the next couple of days
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* https://github.com/cucumber/cucumber-js/releases/tag/v8.5.1
*Thread Reply:* No worries! Let me know if that makes your issues on CodeBuild go away. The team at AWS kindly added this to their backlog, but if it’s fixed from our side I can tell them not to worry.
*Thread Reply:* I’ll try updating our project now, I’ll let you know what happens
*Thread Reply:* This overlaps with https://github.com/cucumber/cucumber-js/issues/1393 - there’s a change being worked on now where BeforeAll would have access to world parameters.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* In the meantime probably the least worst thing to do is to use actual environment variables I.e. set them outside of the cucumber process
*Thread Reply:* Welcome @UoB software developer 👋
You’re right that profiles is a good tool for this kind of thing. You might have your configuration file be like:
{
"chrome": {
"worldParameters": {"browser": "chrome"}
},
"firefox": {
"worldParameters": {"browser": "firefox"}
}
}
And then pick that up in a custom World class that spins up your browser instance:
if (this.parameters.browser === 'chrome') {
// spin up chrome
}
// etc
*Thread Reply:* @David Goss Thanks for the response. Great tip. I'll try this!
*Thread Reply:* babel.config.json
{
"presets": ["@babel/preset-env", "@babel/preset-typescript"]
}
*Thread Reply:* cucumber.json
{
"default": {
"publishQuiet": true,
"requireModule": [
"@babel/register"
],
"require": [
"steps/****/**.ts"
]
}
}
*Thread Reply:* package.json
{
"name": "@ucx/cucumber-screenplay",
"version": "1.0.0-beta.3",
"description": "Cucumber Screenplay support libraries",
"main": "dist/index.js",
"type": "module",
*Thread Reply:* tsconfig.json
{
"files": ["src/index.ts"],
"compilerOptions": {
"target": "esnext",
"module": "commonjs",
"moduleResolution": "node",
"declaration": true,
"outDir": "dist",
"esModuleInterop": true,
"strict": true,
"typeRoots": ["node_modules/@types", "./src"],
"allowSyntheticDefaultImports": true,
"resolveJsonModule": true
},
"include": ["src/****/**"],
"exclude": ["src/****/**.spec**"],
*Thread Reply:* Do you have details of what goes wrong e.g. stack traces?
*Thread Reply:* At first glance I notice you’re using requireModule and require in your config and your module format in tsconfig is commonjs, but then you have type: "module" in the package.json - this could be an issue but not totally sure.
*Thread Reply:* Starting over (ignore the above, I've been editting). This is the script invocation in the package.json:
"scripts": {
"test": "cross-env NODE_OPTIONS=\"--loader ts-node\" cucumber-js",
*Thread Reply:* Currently using a cucumber.cjs to configure cuke with this:
module.exports = {
default: {
publishQuiet: true,
requireModule: [
"ts-node/register"
],
require: [
"steps/****/**.ts"
]
}
}
*Thread Reply:* tsconfig.json:
"compilerOptions": {
"target": "esnext",
"module": "esnext",
"moduleResolution": "node",
"declaration": true,
"outDir": "dist",
"esModuleInterop": true,
"strict": true,
"typeRoots": ["node_modules/@types", "./src"],
"allowSyntheticDefaultImports": true,
"resolveJsonModule": true
},
*Thread Reply:* And when I try to run: ```PS C:\Users\tendo\Projects\ucx\mfa\tests\cucumber-screenplay> npm test
> @ucx/cucumber-screenplay@1.0.0-beta.3 test > cross-env NODE_OPTIONS="--loader ts-node" cucumber-js
(node:24552) ExperimentalWarning: Custom ESM Loaders is an experimental feature. This feature could change at any time
(Use node --trace-warnings ... to show where the warning was created)
Error [ERRREQUIREESM]: Must use import to load ES Module: C:\Users\tendo\Projects\ucx\mfa\tests\cucumber-screenplay\steps\init.ts```
*Thread Reply:* ```import { defineParameterType } from "@cucumber/cucumber";
defineParameterType({ name: "listener", regexp: /[a-zA-Z]+/, });
defineParameterType({
name: "shouter",
regexp: /[a-zA-Z]+/,
});``
Works same.transform` not doing anything here.
*Thread Reply:* ```import { defineParameterType } from "@cucumber/cucumber";
const names = ["listener", "shouter"];
names.forEach((name) => { defineParameterType({ name, // name of the parameter type regexp: /[a-zA-Z]+/, // regular expression to match the parameter }); });``` More DRY. 🤓
*Thread Reply:* You can access this from the transform function too, so you can set this.listener or this,shouter in the parameter type, if you want to.
*Thread Reply:* The idea of the transform function is to turn the string from the regexp capture into something more meaningful and useful in your domain,
*Thread Reply:* If I require instead of import I get a different error.
*Thread Reply:* The only way I've been able to get this to work is to compile the cucumber steps to the file system manually instead of JIT.
*Thread Reply:* The problem seems to be there is no importModule directive that parallels requireModule
*Thread Reply:* I m using Babel and using import without issue. LMK and I can share some snippets. I m using Jest with 🥒
*Thread Reply:* Maybe we could work towards a new example in https://github.com/cucumber-examples/cucumber-js-examples ?
*Thread Reply:* This example uses ts-node, perhaps you two could put together a working example for typescript and babel?
https://github.com/cucumber-examples/cucumber-js-examples/tree/main/examples/typescript-node
*Thread Reply:* Yeah. Possibly. M not using TS, but I will post back with something.
*Thread Reply:* > The problem seems to be there is no importModule directive that parallels requireModule https://github.com/cucumber/cucumber-js/blob/main/docs/esm.md#transpiling
*Thread Reply:* I've been quiet cause I'm still fighting with this. When I have a solution I'll look to helping communicate it.
*Thread Reply:* I'm inexperienced with typescript, so I don't know how much of my problem is that inexperience.
*Thread Reply:* Well, I might suggest leaving TS alone and just focus on the JS part first. Certainly, TS is important, but maybe only pursue one at a time for learning.
*Thread Reply:* I've already built a testing framework up in just JS and maintained it for nearly 2 years now. I want to incorporate TS because we are migrating the rest of the codebase to TS
*Thread Reply:* @matt-SmartBear Still looking into getting the example stuff together. It would be after Labor Day (about 2 weeks or so) before I get to it.
*Thread Reply:* Nice one @Manav Misra - I've been on vacation until now myself. If you want to pair on it, hit me up at calendly.com/mattwynne/
*Thread Reply:* Crazy busy - looking for new position and such...but being as this is Hacktoberfest, I will carve 🎃 out some time for this. 🤓
Hi, I have issue with using cucumber expression. I get the step defn not defined. here is the step definition https://github.com/LayMui/RNLogin/blob/master/tests/src/step-definitions/azureADAuth.steps.ts#L10 and the issue seen https://github.com/LayMui/RNLogin/blob/master/Issue.md thanks
<a href="https://github.com/LayMui/RNLogin">LayMui/RNLogin</a>
<a href="https://github.com/LayMui/RNLogin">LayMui/RNLogin</a>
*Thread Reply:* If you use {string} in your expression, then the value has to be enclosed in double quotes. For matching without quotes, try using {word} instead:
*Thread Reply:* 👋 the readme should get you to a simple hello world that runs and passes https://github.com/cucumber/cucumber-js#readme
Is your question more around how to automate against the React app? Are you thinking of testing at the component level (eg with testing-library), or working it from the outside with a browser driver?
*Thread Reply:* Mostly I'm looking for a starting point for outside in BDD style in a create-react-app
*Thread Reply:* Okay so here's a starter you might find useful https://github.com/davidjgoss/cucumber-puppeteer-typescript-starter
*Thread Reply:* There are some more current ones from others that use playwright not puppeteer, which I think is an improvement.
*Thread Reply:* i would prefer Gherkin over native JS stuff
*Thread Reply:* wouldn't it be possible to build a template for create-react-app?
*Thread Reply:* See https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/timeouts.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* We have an issue for this with an agreed way forward in https://github.com/cucumber/cucumber-js/issues/1153#issuecomment-657899146 but it hasn’t been implemented yet. There’s some other work in flight relating to test run hooks, after that’s done I’ll probably pick this up next (but if you feel like giving it a go in the meantime, go ahead!). There are some workarounds mentioned by other users in the ticket which may or may not be useful.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Hi @David Goss that's great to see some work has been done already. Not sure what I can help with. I may need a clear guide. Thanks.
*Thread Reply:* @Tim Yao if you want to pair, https://calendly.com/mattwynne and we can figure it out together
*Thread Reply:* Thanks @matt-SmartBear ! looks like the timezone for us to meet is quite hard
*Thread Reply:* probably like Melbourne afternoon, seems we are both working.
*Thread Reply:* I'm on PST, so my guess is it's more your morning / my afternoon?
*Thread Reply:* I am also facing the same issue, I want to run some tests in before all and after all in one thread in parallel, kind of one time setup in suite level, Can anyone help me to sort this issue.. Thanks..
*Thread Reply:* What are you trying to achieve?
BeforeAll hooks will run at the start of a test run always. It doesn't matter which file they are defined in and/or if any of the running steps are in the same file.
*Thread Reply:* It sounds like you might want a Before hook which can be targeted at specific scenarios via tags
*Thread Reply:* I have two feature file and different step definitions according to feature I'm running. Now the issue is in each step definition I have defined the before all tag. Unfortunately a weird issue is occurring where it runs the before all , of all the step definition.
When I commented out one before all hook I don't see the above mentioned issue.
*Thread Reply:* Like I said, BeforeAll hooks aren't tied to steps at all. They always run regardless of which scenarios you've filtered to.
*Thread Reply:* This issue got resolved after I used tagged hooks. Thank you all
*Thread Reply:* Do you mean each scenario annotated?? If yes , no I'm not looking for that yet.. The scenario is I have multiple feature file… running one feature file calls all before hooks from different feature file
*Thread Reply:* Seems your answer will solve my question… as I read some issue and solution suggested was to “ if I want to run the hook only for certain feature, need to use conditional hook”
*Thread Reply:* Please give some examples as I don't understand how to tackle this
*Thread Reply:* Hello, your question has actually nothing to do with cucumber itself. The answer depends on your assertion library....
*Thread Reply:* Hello , I do understand it’s about assertion and I don’t know how to assert for the exceptions. Can you help me on this?
*Thread Reply:* Chandrika, I cannot help you as I dont have any detail information about what assertion library you use. Check the docs of your asserting library and of the tool you use for instructing the browser.
*Thread Reply:* Good, so look for the answer here: https://www.chaijs.com/api/
*Thread Reply:* Check out https://github.com/cucumber/cucumber-js/blob/d46c797aba67d36b5f446d202667d9d554b81462/docs/javascript_api.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Looks like babel needs a bit of extra config to work with ts files, I raised https://github.com/Hajime-Suzuki/cucumber-ts-issue/pull/1 which makes it work.
<a href="https://github.com/Hajime-Suzuki/cucumber-ts-issue">Hajime-Suzuki/cucumber-ts-issue</a>
*Thread Reply:* the solution that I found till now is to make a different configuration file for each layer/type of steps for example cucumber.domain.json for domain model and cucumber.acceptance.json for application services
*Thread Reply:* A single configuration file with profiles would probably work https://github.com/cucumber/cucumber-js/blob/main/docs/profiles.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Do you really need two separate clocks? I’m not sure this is possible from reading the Sinon docs, as that install method patches some global methods like setTimeout.
*Thread Reply:* Assuming you can share one clock, just use a single set of Before/After hooks and have them set/use this.clock
That will set the clock property on the “world” which is this in each of your steps.
*Thread Reply:* It might be better to put those hooks in a file like features/support/clock.js rather than a step file, though that’s up to you. Cucumber just loads all of those files and how you organize your code in them is your choice.
*Thread Reply:* Thank you for your reply 😀 Let’s say I want to test 2 different scenarios where the date is in the past and future.
When I use this.clock to set date for the 2 cases in the Before, are those dates isolated? I observed Before was called for each step definition file in parallel. I might see the future date in the test where I need the past date.
I’m very new to cucumber, and I miss important concepts. But this is what I’m struggling with now
*Thread Reply:* My understanding is that when Cucumber runs scenarios in parallel, they run in isolated processes so you won't get any clashes.
Can you give it a try and let us know how you get on?
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Hi @matt-SmartBear, sorry my slow response. I tried to run my tests in parallel but this was not isolated 😞 Here is a reproduction of my issue. https://github.com/Hajime-Suzuki/cucumber-ts-issue
The documentation says BeforeAll and AfterAll are isolated. Maybe Before and After are not.
*Thread Reply:* you can move the logic to function and call the function from both steps
*Thread Reply:* Hello, if the selectors do change, then of course you need to update your tests. For this reason you should have good selectors strategy and collaborate with the devs on this.
*Thread Reply:* Hi, Thank you for your response. We mostly use xpath as selector that includes classnames/ids which sometimes gets changes. Do you have any other strategy?
*Thread Reply:* Xpath is usually not the best approach, have a look here :https://playwright.bootcss.com/python/docs/selectors#best-practices This should give you an idea
*Thread Reply:* No, because when that hook eval's there's no world yet.
*Thread Reply:* We're working on it. See https://github.com/cucumber/cucumber-js/pull/1770
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* The order that hooks are run in is based purely on the order they were defined in, and tags don't have any bearing on that. If the hooks are in multiple files, the order the files are loaded in will influence it. It's pretty common for people to group all their hooks together in one file to remove ambiguity around ordering.
~Hi everyone!~ ~I unfortunately did not realise this channel existed. If anyone could have a look at my question below, it would be much appreciated~ 🙏
https://cucumberbdd.slack.com/archives/C60TKS3SL/p1675330636636339
See the last comment on that message, thank you all!
*Thread Reply:* The worker is env variable won’t be present because the custom assignment code is running on the coordinator not the worker.
Putting the code to one side, could you say more about what you’re trying to achieve here?
*Thread Reply:* Hey @David Goss , Thank you for looking at this
My case is simple, I want to run certain scenarios per specific runners. Example
@Worker_1 Scenario …
@Worker_2 Scenario ...
I’ve made some changes to the source code which work for me (and internally I’ll simply patch-package) and I’ve opened a draft PR in the cucumber-js repo (I’ll update it more tomorrow)
This also covers the use-case that folks have been asking frequently in the cucumber-js issues of running features per worker
Looking forward to hearing your thoughts
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* > My case is simple, I want to run certain scenarios per specific runners. Sorry as I know it can be annoying, but: why do you want to do that? What happens when you just use parallel normally?
You mentioned the “BeforeFeature” request, I wonder if it’s related to that.
*Thread Reply:* Our use-case is a bit of a specific one, but I'm happy to share 🙂
Our web-app has a layer of authentication with a third party provider, once you login then you select different profiles. The active profile is stored per account (so you can't have 2 profiles in the same account active simultaneously).
Therefore, if I just run parallel what happens is that the tests fail since worker A is using Profile A, then worker B switches to Profile B and so worker A loose the profile mid-test and fails...
From what I saw in the github issues, getting access to the worker information was asked a few times and so I figured I'd make the PR (I'll need to fix the tests but the basic idea is there for discussion).
I wouldn't necessary just pass the worker information but rather all worker/world information but haven't done that yet!
Anyway, if this goes against the structure that you are looking for, that's fine and thank you for reading this/maintaining cucumber-js!
Ps: I have a feeling we are on different timezones, so apologies for the delay in responses!
*Thread Reply:* So your plan is to have each worker create its own profile?
*Thread Reply:* Where/when in your test suite does the login happen? Is it per scenario, or in some kind of global setup?
*Thread Reply:* Heya @matt-SmartBear , It's done inside each feature file, in the background definition https://cucumber.io/docs/gherkin/reference/#background
Example, main-page.feature: ```@WORKER1 Feature: Main Background Given I login in as 'account1'
Scenario: Can see the profiles for choice ...``` Other github issues which I think match the same as I'm asking: https://github.com/cucumber/cucumber-js/issues/2222 https://github.com/cucumber/cucumber-js/issues/2217 https://github.com/cucumber/cucumber-js/issues/1044
If you don't mind me asking, is the parallel implementation in cucumber-js, parallelization or concurrency? I'm seeing that workers sometime idle while the others run, so I'm just making sure if it's my setup
Thank you for looking 🙂
*Thread Reply:* OK, so in your context the problem you want to solve is to not have any two scenarios running simultaneously with the same account profile?
*Thread Reply:* Kinda, yes
I don't want to have two scenarios running the same account and different profiles (each account can choose from multiple profiles but can only have one active at a time)
E.g. Scenario 1 • Mat account ◦ Admin • Scenario 2 ◦ Regular user Once (2) runs, it takes over the profile and (1) fails
Could you please share if you know if the cucumber-js implementation is parallel or concurrent?
With the code changes I did to cucumber-js it does run as I want it but sometimes I can see some workers idling for a bit while the others run. Then they continue later
Hope that was clear and thank you for looking at this
*Thread Reply:* > parallel or concurrent I'm afraid I don't know the difference. Can you explain?
*Thread Reply:* I don’t think it translates neatly to either, but maybe it’s more like parallelism because the workers are independent threads. It’s possible for some of them to be idle if there are fewer scenarios still to be executed than available workers and/or if the setParallelCanAssign logic is constraining what can be assigned.
*Thread Reply:* Reading your explanation, I’m struggling to see why the current functionality couldn’t work for you. If users are the constrained resource, then tag the scenarios with those eg. @User-Matt and then use atMostOnePicklePerTag to ensure that user is only ever logged in from one place at a time. The fact that it might change workers between scenarios shouldn’t matter - you login at the start of each scenario, right? I think your focus on keeping it all on the same worker might be a little misplaced.
*Thread Reply:* You could even do a more custom function based on the combination of user- and profile-related tags if that would make it more efficient. But I don’t think you need to assert which worker.
*Thread Reply:* That's fine, I'll update the PR soon to fix the tests and make it reviewable
You can then decide wether you want it or not, but it should help other folks who've asked for use-cases like this (or at least they can patch-package it if they want)
Thank you for looking at this!
*Thread Reply:* I've updated the tests to the best of my knowledge and removed the draft
If you could share your thoughts there, that would be much appreciated
I can understand if this goes against the maintainers long-term plan, and if that's the case I don't mind at all if this gets closed.
Either way, thank you all for looking and sharing your thoughts 🙏
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* “Glue” generally refers to the code that called by your step definitions that drives your automation. I think the term is used more in the JVM world. So JS has it but just doesn’t call it that - we refer more to “support code”
*Thread Reply:* I suspect the problem is Nextjs insists that the project be compiled to ESM while at the same time not having the project marked as type="module"; and if I do add that to the package.json file Next.js blows up - so I need to figure out how to do this with Cucumber alone first then approach that hurdle.
*Thread Reply:* And the docs are spot on. So now to bring in the tsconfig directives from nextjs and find the culprit
*Thread Reply:* Solution. NextJS' tsconfig.json.compilerOptions.module directive was set to "esnext" and Cucumber needs it to be "commonjs"
*Thread Reply:* The docs need to mention this somehow. I'll look into a phrasing and make a PR later today on it.
*Thread Reply:* Hi @Michael Morris. I am not fluent in JS, but with many years working around it I am trying to make sense out of it. Just by seeing your message, I hope to get some more understanding about it.
I wanted to use cucumber-js with React app by touching it on sort of a unit level, so not going through a webdriver, but really to write living documentation for business rules of a few complex components. So, I need **.tsx extension for steps, but when extension of file is changed, cucumber-js ignores that file entirely. I struggled to understand the whole thing about modules and transpilers, and one of JS developer wasn’t able to help me, so I decided to give jest-cucumber (https://github.com/bencompton/jest-cucumber) a try.
The result is that it works really great, especially with automatic step binding alternative and it’s incorporated in Jest tests, and testing library or any other tool can be used just out of the box as well, but reporting feature is broken so the point of having a widely accessible documentation is completely missed.
To be honest, I would love to have it all implemented just in cucumber-js but not sure if this is feasible, and did you make it working, or there is a need to request that feature?
This is the example where I cannot make it working for React components within features/support/steps.tsx file:
```import { Given, When, Then } from '@cucumber/cucumber'
...
When('flight summary component is rendered', function () {
render(
<Container>
<FlightSummary flight={flightDetails} />
</Container>
);
});``
Intsconfig.json, module is set toesnext`. I guess it should stay like that in my case, as the application already runs with that setting?
Thanks for reading it!
*Thread Reply:* P.S. I followed instructions in your PR, however, I wasn’t able to make it working for my case using a recipe for ts-node. Still, **.tsx file is being ignored so suggestion to add all steps is provided by cucumber-js.
*Thread Reply:* ts-node doesn't work with tsx files to my knowledge - you'll need to use Babel as the transpiler.
*Thread Reply:* That does seem odd. I’ll see if I can reproduce but might have to come back to you for more information.
*Thread Reply:* Relatedly, since it seems like you’re triggering Cucumber from some code of your own, have you seen the JavaScript API? It doesn’t currently have a way to stop the test run mid-flight but we’re intending to add that soon. https://github.com/cucumber/cucumber-js/blob/main/docs/javascript_api.md
I’d be interested to hear about your use case for running Cucumber from code and wanting to stop the run mid-flight, if that’s something you’d be okay with sharing.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Hi David,
Thanks for your quick response! I'll spend some time and try out the API, I didn't know about it. 🙂 Regarding the stopping of the execution - it's more like I want to be able to stop the test suite execution upon a some signal, not the test itself (because there could be some unexpected behavior, if the test does not finish its work - database interactions, etc.) Since, every test should not be dependent on another, stopping the suite execution should not cause an issue. I'm running the cucumber through the code, because we developed a service, which is responsible for executing the test suites on some remote machines. For example, if the service receives a POST request with some test details, it's going to start the corresponding execution on specific machine. So I want to be able to stop the test suite execution upon a some request to my service.
Hello Cucumbers, I am looking for some clarification and I hope I can find some help over here regarding testing React components on - let’s call it - unit test level so without a webdriver. I have shared more details in this thread: https://cucumberbdd.slack.com/archives/C6QJ6N695/p1677093082738129?thread_ts=1676484576.091459&cid=C6QJ6N695
*Thread Reply:* I'm sorry I'm so late in seeing this - I only check this board once every few weeks or when I'm actively developing. When you transpile modules you have to import the steps using a glob pattern that explicitly mentions their extension. I have typescript set to export as commonJS during testing so this is my cucumber config file for a NextJS site
module.exports = {
default: {
requireModule: ['ts-node/register'],
require: ['features/steps/****/**.ts'],
publishQuiet: true,
exit: true
}
}
*Thread Reply:* You should be able to get it to work by changing the ts above to tsx and using the import directive if you exporting to a module. NextJS isn't happy having the package.json set to module mode for some reason so I had to go with require.
*Thread Reply:* Thanks @Michael Morris! I asked a JavaScript developer some time ago but I didn’t have time to check it out. Indeed, he suggested me to try the following:
> you might be able to use tsc to translate tsx to jsx and then use babel to translate jsx to JS. I will try to sort it out soon, so will get back to you in any case. Hope this can work :fingerscrossed: .
*Thread Reply:* No, if the TSX module is installed Typescript can translate TSX straight to JS.
*Thread Reply:* https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/world.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Not currently possible, but something we could look at adding via formatOptions. Would you mind raising a feature request at https://github.com/cucumber/cucumber-js/issues/new/choose
*Thread Reply:* In ruby I know we cannot set the name of the files, but we can set the name of the directory. Not sure if this helps or not @David Goss (I think the filenames are semi fixed, cos of reporter tools upstream).
*Thread Reply:* There’s no technical maximum from Cucumber’s perspective. The limiting factors will be the hardware you run on (the more you do in parallel, the more memory will be consumed by each worker), and the ability of your system under test to handle the parallel activity (shared resources, dependencies between scenarios, etc).
*Thread Reply:* Thanks for your response, I am running the scripts on the docker container hosted in kubernetes cluster, I see only till 4 thread, the results are stable
*Thread Reply:* Have you seen https://github.com/cucumber/cucumber-js/blob/main/docs/parallel.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* Yup, actually i found it after posting the question, but i did not get this part
*Thread Reply:* There is nothing in the language that prevents this, and further you can actually pull it off in userland since JavaScript allows functions to be overwritten. I'm open to developing this into core if the core team approves, but I need to know a bit more on this - specifically what are the use cases?
*Thread Reply:* Have you had the need to reuse a set of feature tests? And allow others to extend the step definitions when they need?
*Thread Reply:* Not really, because you're kind of attacking the problem incorrectly.
*Thread Reply:* Two situations likely will occur. You want a standard cucumber cuke -> Where things like aruba are a massive help
*Thread Reply:* Or you want a situation where you want the "first bit of" cuke 1. Or the "middle bit of" cuke2. In that situation you are incorrectly storing reproducible information in the cuke, where it should live in a class/helper/OO object.
*Thread Reply:* I have a test suite, I need to use it to test multiple projects. However, a few projects have some customisation which makes all tests fail. If I can override one step definition for that project, then it should be fine.
*Thread Reply:* aaaah oki, so you have similar to cucumber. golden features?
*Thread Reply:* so in cucumber, we have a concept known as golden. Which is where we have an input and output, and we put a different "engine" in the middle.
The test is that with the same input, and a different engine, we "should" get the same output.
*Thread Reply:* I can't remember the exact output. But it's on github somewhere.
*Thread Reply:* https://github.com/cucumber/gherkin/tree/main/testdata is one example for gherkin
*Thread Reply:* So
Feature: Inconsistent cell counts
Scenario: minimalistic
Given a data table with inconsistent cell count
| foo | bar |
| boz |
Scenario Outline: minimalistic
Given the <what>
Examples:
| what |
| minimalism | extra
*Thread Reply:* should always produce
{"parseError":{"message":"(6:7): inconsistent cell count within the table","source":{"location":{"column":7,"line":6},"uri":"../testdata/bad/inconsistentcellcount.feature"}}} {"parseError":{"message":"(14:5): inconsistent cell count within the table","source":{"location":{"column":5,"line":14},"uri":"../testdata/bad/inconsistentcellcount.feature"}}}
*Thread Reply:* Let me think...not exactly same situation but kind of...
*Thread Reply:* I can always handle them in a centralised step definition file, by checking what test target is, then handle them differently. But in my case, I am looking for a distributed way to allow it to be handled in each project.
*Thread Reply:* It's like Interface , I want each project can have their own implementation, maybe
*Thread Reply:* But I have a same test scenario, like what your example shows.
*Thread Reply:* yes so you want either screenplay pattern, or you have situation2 where you're not using common helpers/classes
*Thread Reply:* Hi @Tim Yao I am curious why would you need to "reuse and extend" step definitions ? For me they describe a very precise business logic and should in theory be unique per project
*Thread Reply:* reusing step defs is quite common if they're generic ones. Which is what aruba provides users with
*Thread Reply:* ah sorry I thought you were talking about the gherkin itself
*Thread Reply:* @Aleksandar KIRILOV It's actually is reuse feature file...Like the example of the golden feature, I want to reuse it to test multiple projects. The business logic in the test steps are same, but each project may have different implementations for certain steps, and they are agnostic to me.
*Thread Reply:* Each project can have their own step definitions. Those can call common support call to prevent code duplication.
*Thread Reply:* You could also have one common set of step definitions for those steps that do not need redefinition between patterns.
*Thread Reply:* The more you describe the problem, the more it seems this can be steered around.
*Thread Reply:* Thanks, Those can call common support call to prevent code duplication. not sure how to do that @Michael Morris. Can you please give more details?
*Thread Reply:* Given this directory structure
/steps
/project1
/project2
/project3
/common
/support
You could then load the step definitions for project1 with cucumber-js --require /steps/project1/****/**.js --require /steps/common/****/**.js
*Thread Reply:* Then your project step definitions could import from /support and extend those methods or do some config method before calling those methods.
*Thread Reply:* There's a multitude of paths to go through here. Without knowing more specifics I don't know which ones are best.
*Thread Reply:* OK. I got that. It does solve some of the cases. But I can't have one js file for each definition. Using require load way, I can't exclude one step definition from one file which contains other steps I need.
*Thread Reply:* I am also thinking about creating abstraction inside the step definition that may need to be extended. Inside that step definition, use JS to call a function that will load code dynamically which can allow project to override the default code.
*Thread Reply:* require can be invoked multiple times (so can import)
*Thread Reply:* Use your cucumber config file to set up a profile for each project
*Thread Reply:* And have the profile load the definitions needed by that project.
*Thread Reply:* Thanks. Here is what I think is a work around:
- split the step definitions that need to be overridden into a separate file.
- Then use the profile to include/exclude that file.
- In different projects, chose the profile they need. I think it will work as long as the customisation needs are small, which is what we are. Should be ok.
*Thread Reply:* I’m not aware of a puppeteer community.
Have you tried using a template literal for the input string?
*Thread Reply:* @plocket you can use page.evaluate and it becomes just javascript.
*Thread Reply:* Hmm, that might be needed when using .value, but this problem may be in combination with cucumber - when I manually code handler.type("Foo\nBar"), I do get multiple lines in the text area, but when I use the value that cucumber passes in as an argument, I don’t get any new lines, just literally “Foo\nBar” in the text area. (Example step: Then I type "Foo\nBar"). Is cucumber stringifying the argument first?
*Thread Reply:* I think doc strings is what you need https://cucumber.io/docs/gherkin/reference/
*Thread Reply:* That’ll work in most places for us, but sometimes it needs to be in a table.
*Thread Reply:* It needs to work in a table as well, not just as a doc string. Any ideas?
*Thread Reply:* Hello @plocket just use JS string.replace() to replace your escaped \n by \n - this is mostly code related at this point not gherkin issue imo
*Thread Reply:* ~Thank you. I was having trouble understanding how cucumber was processing the string. I tried to experiment to find out, but kept getting tripped up by the backslashes. How do I make sure to unescape other characters? I’ve been trying to look it up online for a few hours now. The most I could find was~
.replace(/\\{2,}/g, '\\')
~But I experimented with that and it didn’t work at all. I did 'foo\\nbar'.replace(/\\{2,}/g, '\\') in the console and it just came back as 'foo\\nbar' .~
*Thread Reply:* Instead of that, can you point me to where in the cucumberjs code the string is being escaped? That way I can better try to unescape it. I’ll ask on js channels about that.
*Thread Reply:* Is it something like JSON.stringify or is it a Markdown parser?
*Thread Reply:* You may be looking for https://github.com/cucumber/cucumber-expressions
*Thread Reply:* Thanks! Do you know where in there the “stringifying” is going on?
*Thread Reply:* We use console.log() in our AfterAll() hook and that works fine if that’s what you mean.
*Thread Reply:* it logs all the logs that have been logged during the run and through all steps?
*Thread Reply:* We add those to a string variable as we go and print the whole thing at the end.
*Thread Reply:* That said, if you console log during the tests, that shows up too. I think there is something you can do with the… formatter (?) that can influence the output. Also, we create a file we call a “report.txt” that contains all our logs. [We use that same string that I described.]
*Thread Reply:* you externally created a file for logs and added it to reports, or sth already built-in?
*Thread Reply:* We currently don’t add it to the “published” report. We just add it to our directory and .gitignore it.
*Thread Reply:* and maybe i was not clear in asking my question; what i meant in my question is I need to add the logs to the published report
*Thread Reply:* I think I’ve seen folks talk about something called the formatter. Do other folks know if that’s right? Would that affect the published report? (posting to channel)
*Thread Reply:* There is this.log() which is just shorthand for doing a plain text attachment - not sure if that helps @Shatha
*Thread Reply:* is it something that can help in having all of the logs at the end of the run? or attached in the report folder? and if so, can you help me in how to apply that?
*Thread Reply:* I’ve done this by handling the logic internally - I abstract the functions that I define in a Step. Then I catch certain types of errors and call the function again if appropriate.
*Thread Reply:* The doc says it is not possible:
https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/hooks.md#beforeall--afterall
Can you tell us more about your use case ? If you need to access the world inside BeforeAll maybe you are trying to use it in a wrong way
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I want to connect to Database and common setup (one time setup in test suite level) in before All method in parallel execution, If i connect to DB in before method , I have to create the data setup for every scenario, Data is creating in runtime, so it cannot interact with the other threads which is running in parallel, so I have to create in before all
*Thread Reply:* Do you need the world object to do such technical things like connecting to a database ? Just put your SQL in a file and load it in the "BeforeAll"
Also imo it is a good practice to have separate dataset for each test - this way you get better isolation and your tests will not leak.
*Thread Reply:* I have different DB env and based on the world parameter env , I want to connect to that DB
*Thread Reply:* It is having application constraint that only one record created for Admin and I want to access the admin account after creating it in one thread
*Thread Reply:* I think you should create a second tsconfig.json for your test that has the appropriate includes and excludes and then you should specify that ts-node in your tests use that tsconfig. Not exactly sure of the syntax though - sorry
*Thread Reply:* You can read the https://github.com/cucumber/cucumber-expressions#alternative-text
If you need full control then use regex expressions.
PS: Imo you should write your step as follows:
Then I send a request to Web and NA property api ...
Reuse code in your glue and keep your feature files clean and readble.
*Thread Reply:* Also gherkin is about business logic and you seem to be testing some technical APIs - maybe you should take a look at karate
*Thread Reply:* this medium article helped me to fix the issue: https://medium.com/taqtilebr/testing-front-end-applications-with-cucumber-and-cypress-9666771bc8c4
and this step definition is working fine in my case:
Given(/I send a request to (NA|Web) search api/, (api) => {
*Thread Reply:* In cucumber expression, you also can use custom parameter type https://github.com/cucumber/cucumber-expressions#custom-parameter-types But it adds extra code which defines your reusable type, like: ```import { defineParameterType } from '@cucumber/cucumber'
defineParameterType({ name: 'propertyName', regexp: /Web|NA/, transformer: s => s })```
*Thread Reply:* Something like this? ```let username; let password;
BeforeAll(function () { username = random(); password = random(); });
Then(...() { loginwith( username, password ); });```
*Thread Reply:* From what I just read about BeforeAll when running in parallel (just below this), this solution won’t work if you’re running tests in parallel. Maybe you’d define them right at the start:
let username = random();
let password = random();
*Thread Reply:* I haven't used parallel testing but I think there is a variable set in the process.env that tells you the executor ID. You might use that to do something like
if(process.env["CUCUMBERWORKERID"] === 1)
*Thread Reply:* Can you tell us what is your use case why do you need it to run only once ?
*Thread Reply:* so i am building api tests and i need before running the tests to create users per supported countries to be used in my test
*Thread Reply:* > if(process.env["CUCUMBERWORKERID"] === 1) this one to be set in beforeAll right
*Thread Reply:* https://github.com/cucumber/cucumber-js/blob/main/docs/parallel.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* just before you go: as a best practice each test should have it's own data set and cleaned at the end to avoid leaking so what you are doing is a bit a "quick and dirty"
*Thread Reply:* IMO each test should create the user in the BeforeAll and in AfterAll should clean them
*Thread Reply:* this will slow down a bit your tests but will make them more stable
*Thread Reply:* but the thing in my case, there is no value of creating new users for each test because i am creating them for the sake of testing the API, that does no write or modifications
*Thread Reply:* Instead of doing this instead you should be using a more model based approach. I've done this in my last few jobs and I'm going to make a gem based off this I think going forward
Your tests should be doing something like
Given usertype A
This then does a findorcreate by "logic"
*Thread Reply:* Hmm, did not get your point fully mind sharing an example!
*Thread Reply:* ```Scenario: A user who likes apples can look at a shopping list Given a user who is an apple fan Then they are able to open the shopping list
steps
Given('a user who is an apple fan') do # logic to find or create a user who is an apple fan end```
*Thread Reply:* There is a open issue https://github.com/cucumber/cucumber-js/issues/1153
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* PR but no progress https://github.com/cucumber/cucumber-js/pull/2140
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* My workaround is use another script to run the required test condition setup before I run the cucumber-js testing. I pipelined them in CI. I think it's also easy to setup in npm script as well.
*Thread Reply:* it would be something like (Say in jenkins)
```stage('prior setup') { steps { sh 'run your step here' } }
stage('cucumber') { steps { sh 'bundle exec cucumber' } }```
*Thread Reply:* @Tim Yao I assume you're the same person from the issue/PR. But in essence I've marked the issue as won't fix, and I advise the PR to go the same route. It's solving a problem that in essence isn't broken in my eyes. But I'll let other people maybe provide more input. You've even articulated here a better solution which I wholeheartedly agree with.
I appreciate you may have spent some time on the original PR, and as such won't close that (Plus I'm no JS expert). But either way it's great to see you contributing and please don't stop on my behalf.
*Thread Reply:* Issue resolved by upgrading selenium Webdriver and geckodriver
*Thread Reply:* I’m not sure about if I can understand the problem: do you have to provide Gherkin Scenarios in different languages or does your application support multiple languages or do you have to test your application for multi-language environments (eg., browsers, OS’s)?
*Thread Reply:* The applications support multiple languages and each language needs to be tested independently, @Matthias Carell. Without some special behavior, each language would need its own Gherkin Scenarios that were otherwise identical to those of the other languages. Does that make more sense?
*Thread Reply:* I see. I think I would use data tables like this: ```Given the user has set the app language to <language_name> And the cancel button shows the corresponding label | lang | label | | en | cancel | | de | Abbruch | When the user clicks on the button Then a message confirms this action | lang | label | | en | Action canceled | | de | Aktion abgebrochen |
Examples: multi language cancellation | language | language_name | | en | English | | de | German |``` This way you can use the same scenario for multiple languages. The Examples section (Outline table) would repeat the scenario for each language while the steps with data tables could pick the actual language setting accordingly.
*Thread Reply:* Wow, I wasn’t aware of Examples. I’ll have to look at how they work. Still some extra work to run languages only conditionally, but we can handle that other ways. I’ll look into this and see if it suits our purposes!
*Thread Reply:* Examples: are part of Scenario Outlines as described here https://cucumber.io/docs/gherkin/reference/#scenario-outline Newer versions of Cucumber do recognize Examples: without starting the scenario with Scenario Outline:
*Thread Reply:* Regarding how to run only selected languages please look at https://cucumber.io/docs/cucumber/api/?lang=javascript#tags You will find an example that have multiple example tables for a single outline scenario. This allows you to add tags to each example table like this: ```Scenario Outline: show feedback on cancel action Given the user has set the app language to <language_name> And the cancel button shows the corresponding label | lang | label | | en | cancel | | de | Abbruch | When the user clicks on the button Then a message confirms this action | lang | label | | en | Action canceled | | de | Aktion abgebrochen |
@langen Examples: cancellation in english | language | languagename | | en | English |
@langde Examples: cancellation in german | language | languagename | | de | German |``` Now you can use tags to run scenarios only for a selection of all available languages.
*Thread Reply:* Thanks, this is a really strong lead and I’ll look into it! I must say there were no docs on cucumberjs about this capability existing in cucumberjs. Is there a doc that describes either that cucumberjs always has the same capabilities as cucumber? Or that lists which features it has implemented so far?
*Thread Reply:* When I say Cucumber I don’t have a specific version in my mind. The docs from cucumber.io/docs should match with the current version of cucumberjs If a feature is language specific the page offers you a filter to show the doc related to the right language:
*Thread Reply:* However most Cucumber versions should support most aspects of the Gherkin language. I would expect all more mature versions (eg. cucumberjs or cucumber for Java) to support scenario outlines. If you want to learn more I would recommend Writing Great Specifications. I learned a lot from this book.
*Thread Reply:* I built a test suite that had multiple language support - 12 of them. The language to be tested was passed as a world parameter. Gherkin steps did not refer to the exact text that would be appearing nor should they - they aren't supposed to contain details that small.
*Thread Reply:* Yes, I agree Gherkin steps should avoid referring to the exact text. We’re making the steps language agnostic generally. The only reason that we need to test with every language is to make sure that the translators don’t mess up the code. That is, we won’t need the tables that check for words from the different languages, though I do appreciate you providing those in the example.
We do have a weird use case, and languages is only one reason, so that we can support vulnerable users. If you ever want to get into it, let me know ;)
*Thread Reply:* That said, our internal tests to make sure we’re handling languages properly will definitely use the example tables you showed to look for phrases so that we can be sure we’re actually testing languages correctly.
*Thread Reply:* ~Quick question: Can the example tables have multiple rows? I’m not sure why you split those rows into separate tables.~ I think I understand - they have different tags.
*Thread Reply:* Are you sure you're not reinventing the wheel? The implementation my team used put all the translations into a json file structured something like this:
{
"color": {
"US": "color",
"UK": "colour",
"other": "color"
}
}
*Thread Reply:* Each major language has a file, and regional variation is covered by the nation codes as seen above.
*Thread Reply:* The test steps can refer to the translation by their key.
*Thread Reply:* The tests then load the same language file the code under test is using. If the code is working correctly it should provide a match.
*Thread Reply:* Javascript has multiple i18n implementations. I wouldn't recommend inventing yet another one - do some research, find an existing solution you like and save yourself a headache.
*Thread Reply:* I can’t control how the translation is arranged. As I said in the original post, it sucks, but life isn’t perfect and that’s what we’ve got.
*Thread Reply:* The code under test should be arriving at it using some process similar to the above. If not sourced from a json file perhaps sourced from an API endpoint?
*Thread Reply:* Whichever is used the testing code should independently fetch the value from the source, then verify the code under test is displaying it.
*Thread Reply:* Each translation string almost certainly has an identifying key. I've never seen a translation system that didn't do this.
*Thread Reply:* Does it matter how the language is being triggered? The end to end test goes to the url for the appropriate language. It’s in the url args.
*Thread Reply:* If you want to discuss the translation situation, maybe we can DM about that?
*Thread Reply:* It shouldn't matter how the language is triggered so long as the testing code has access to the translation strings.
*Thread Reply:* And I'll be happy to take a few DM's. Since I'm between jobs I've got the free time.
*Thread Reply:* if the scenarios proper can be written in english, I would write the scenarios once without outlines, then use a config tool to pass in your dynamic language. It makes writing tests 10x easier.
*Thread Reply:* Use things like hash lookups or yml files to store your i18n translations.
*Thread Reply:* Unless you explicitly want the translations visible in your report.
*Thread Reply:* That sounds like an interesting alternative, @luke, but I’m not quite understanding. Why do I need a scenario outline if I’m not going to use Examples? How/where would I use the config values? Can you give me a small example?
*Thread Reply:* you would write one scenario for each use case, and use i18n to translate
*Thread Reply:* You can then feed in the translation either as a cucumber step or an env var step. Easy-peasy
*Thread Reply:* Not sure how I can make that kind of Step that wouldn’t require writing a test for each language. If I use env vars, I’m not sure how I’d make sure each language had its own unique test.
*Thread Reply:* Because you just lean on the env var in your step/support code and use i18n, it's literally designed for this.
*Thread Reply:* So you either have a given step, or an env var (Which you could set in a step or as a pre-req)
*Thread Reply:* ```Given('The website is in English') do I18n.locale = :en end
Given('The website is in Norwegian') do I18n.locale = :nb end
Then('I am welcomed to the homepage') do expect(@app.home.welcomemessage).to eq(I18n.t('home.welcomemessage')) end```
*Thread Reply:* or for env var you do something like
LANG=en cucucumber
LANG=nb cucumber
*Thread Reply:* What does the single Scenario look like that will run a test for each of English and Norwegian?
*Thread Reply:* ```Feature: Run test Scenario Outline: Validate homepage message Given the website is in <language> And I am on the homepage Then I am welcomed to the homepage
Examples:
| language |
| English |
| Norwegian |```
*Thread Reply:* if you want to use env vars, then it depends if you want them in a step or not.
*Thread Reply:* if you use env vars without steps then it would not have the outline
*Thread Reply:* I think I’m trying to say that I don’t know how I’d do separate tests if I wanted to use env vars.
*Thread Reply:* Feature: Run test
Scenario Outline: Validate homepage message
Given I am on the homepage
Then I am welcomed to the homepage
*Thread Reply:* Yes, and LANG=nb cucumber, but what’s the rest of the code?
*Thread Reply:* so in your env.rb you would set the locale from the env var
*Thread Reply:* Though I’m not sure how you can test both languages at the same time like that.
*Thread Reply:* I don't know your business case. Maybe some tests only are valid in one language, and you don't want to tag your tests as @not_nb e.t.c.
*Thread Reply:* you could then have tasks that ran 1/multiple languages as a single script
*Thread Reply:* ```# commoneuropeanlanguages.sh
echo 'Running english' LANG=en cucumber sleep 1 echo 'Running swedish' LANG=sv cucumber sleep 1 echo 'Running french' LANG=fr cucumber sleep 1```
*Thread Reply:* stages {
stage('Run common languages') {
sh('./common_european_languages.sh')
}
}
*Thread Reply:* Happy to help.
In short, one of two ways will help (IF you don't need the verbose translations visible to your report recipients).
Way 1 - Use env vars, inject them in - preferably silently, run your tests for 1 language Way 2 - create gherkin steps to pivot / switch languages, useful if your tests need to flit and flot between 1/2/3/4/5 languages per scenario / rule e.t.c.
*Thread Reply:* And above all else, please lean on i18n if at all possible 😄 it's literally designed for this exact purpose
*Thread Reply:* Hello, I would suggest to wrap your tests in a JS file and use the cucumber-js API to run your tests and add some logic that evaluates the number of steps ran and calls "process.exit(code)"
*Thread Reply:* https://github.com/cucumber/cucumber-js/blob/main/docs/javascript_api.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* An expected scenarios count as a cli argument isn't a bad idea though.
*Thread Reply:* @Michael Morris or atleast "bigger than zero". Maybe we should open an issue in github ?
*Thread Reply:* @luke please read the full conversation - if missconfigured, cucumberjs can report that everything is OK when infact your tests are not ran at all. It will not alter any existing functionalities to add a simple --fail-on-no-tests parameter
*Thread Reply:* I have done. If the startup is wrong / faulty, cucumber doesn't know this. Again 0 tests being ran - irrespective of why, isn't an issue.
*Thread Reply:* If the tests "failed" to run, then that's a different situation.
*Thread Reply:* @luke sorry my friend. Ok then maybe wrapping the test run in a bash or js script can provide the additional logic that OP needs.
*Thread Reply:* Yeh, I'm not sure readily how easy that is to do / why you'd want to do that, but each to their own.
One way you can do it in ruby (I'm not sure if this exists in other languages), is the concept of a dry run
*Thread Reply:* @luke It is not an expected result for the CI to have 0 tests run. I am not recommending that it always fails, but rather suggesting that a flag be added. This flag would indicate that if 0 tests are run, the expectation is for the run to fail.
*Thread Reply:* Sorry, that's just not the case, I'm not sure why you're thinking this. A simple use case would be
@not_used
Feature: Abc
Scenario: Def
cucumber -t @abc
*Thread Reply:* ```[17:25:53] luke@luke-ubuntu:~/Code/exordo-app-automation$ cucumber -t @abc 2023-06-28 11:21:10 - INFO - Automation Helpers - Adding patch: AutomationHelpers::Patches::SeleniumLogger 2023-06-28 11:21:10 - DEBUG - Automation Helpers - When using the Selenium Logger that is set to pipe to a file, the Net::HTTP adapter (default), can return unencoded binary (confusingly called ASCII-8BIT in Ruby). Consequently we set the logger to binmode so it doesn't try to encode the data - this would always cause errors for non-ASCII characters, whatever the parent encoding is. An example of this is ©.
See https://github.com/ruby/net-http/issues/14 for a root cause analysis of the Adapter
2023-06-28 11:21:10 - INFO - Automation Helpers - Patch successfully added.
0 scenarios 0 steps 0m0.000s```
*Thread Reply:* [11:21:10] luke@luke-ubuntu:~/Code/exordo-app-automation$ echo $?
0
*Thread Reply:* This would fall into the "not-cucumber" pot. If you want to fail based on the number of scenarios ran, that's fine. But it's not something that should be happening by the design of cucumber.
*Thread Reply:* ```# env.rb
frozenstringliteral: true
require 'dotenv' Dotenv.load('.env')
yapowwwww```
*Thread Reply:* cucumber -t @abc
undefined local variable or method `yapowwwww' for main:Object (NameError)
/home/luke/Code/exordo-app-automation/features/support/env.rb:6:in `<top (required)>'
<internal:/usr/share/rvm/rubies/ruby-3.2.1/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
<internal:/usr/share/rvm/rubies/ruby-3.2.1/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
/usr/share/rvm/gems/ruby-3.2.1/gems/cucumber-8.0.0/lib/cucumber/glue/registry_and_more.rb:123:in `load_code_file'
/usr/share/rvm/gems/ruby-3.2.1/gems/cucumber-8.0.0/lib/cucumber/runtime/support_code.rb:145:in `load_file'
/usr/share/rvm/gems/ruby-3.2.1/gems/cucumber-8.0.0/lib/cucumber/runtime/support_code.rb:82:in `block in load_files!'
/usr/share/rvm/gems/ruby-3.2.1/gems/cucumber-8.0.0/lib/cucumber/runtime/support_code.rb:81:in `each'
/usr/share/rvm/gems/ruby-3.2.1/gems/cucumber-8.0.0/lib/cucumber/runtime/support_code.rb:81:in `load_files!'
/usr/share/rvm/gems/ruby-3.2.1/gems/cucumber-8.0.0/lib/cucumber/runtime.rb:274:in `load_step_definitions'
/usr/share/rvm/gems/ruby-3.2.1/gems/cucumber-8.0.0/lib/cucumber/runtime.rb:74:in `run!'
/usr/share/rvm/gems/ruby-3.2.1/gems/cucumber-8.0.0/lib/cucumber/cli/main.rb:29:in `execute!'
/usr/share/rvm/gems/ruby-3.2.1/gems/cucumber-8.0.0/bin/cucumber:9:in `<top (required)>'
/usr/share/rvm/gems/ruby-3.2.1/bin/cucumber:25:in `load'
/usr/share/rvm/gems/ruby-3.2.1/bin/cucumber:25:in `<main>'
/usr/share/rvm/gems/ruby-3.2.1/bin/ruby_executable_hooks:22:in `eval'
/usr/share/rvm/gems/ruby-3.2.1/bin/ruby_executable_hooks:22:in `<main>'
*Thread Reply:* [11:24:32] luke@luke-ubuntu:~/Code/exordo-app-automation$ echo $?
2
*Thread Reply:* Hopefully that clarifies everything for you 😄 But yeh if you want to write some shell wrapper that queries the amount of ran scenarios, that's all well and good. There is a gem called cuke_modeler that will give you those stats. maybe something similar exists in JS or can be written readily?
*Thread Reply:* @luke In normal circumstances, I expect it to behave as it is currently doing. However, if I pass the appropriate flag, I would prefer it would fail fail.
*Thread Reply:* what is the exit code from that command? I would expect it to be exiting with 0.
*Thread Reply:* But it actually failed becuse nothing ran even though It should have about 50 test runing
*Thread Reply:* @luke What I meant was adding a CLI arg
cucumber-js --expected-scenarios 20
If the number of passes doesn't match the argument, exit code 1. And yeah, you can do this with a wrapper but it isn't straight forward.
*Thread Reply:* @Michael Morris i think the better solution for me is --minimum-tests 1
*Thread Reply:* Maybe just --expect-tests and if that flag is present fail if nothing runs.
*Thread Reply:* Hello I have opened an issue "feature request" in cucumber github
https://github.com/cucumber/cucumber-js/issues/2296
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* @Michael Morris I understand what you meant, but I've said several times now this isn't something that should be determined by cucumber.
*Thread Reply:* Background steps are added to all scenarios in the same feature. So the tag would apply to all scenarios. Therefore I don’t think it would make sense to have tags in the background section. Regarding the outline / examples tables I don’t think this is available on feature level. As discussed in the thread before you might be wetter with the setting the language in the configuration and run tests with one configuration per language.
*Thread Reply:* the parser error will tell you the problem, likely here an invalid keyword
*Thread Reply:* Thanks for the input. I’ve tried with the configuration, but I can’t get it to do what I want. What the users want to be able to do:
• By default, only the default language will be run.
• Users would be able to manually trigger running the same separate individual tests for each of multiple languages.
• Users would be able to manually trigger just one language.
I’m not sure how to achieve the second of those objectives using the configuration file since without Examples, a Scenario can only be triggered once. Any ideas?
*Thread Reply:* If you want to dynamically (but not at runtime), configure the tests to be ran multiple times, that's not something you "should" do with cucumber or any testing framework, as it breaks TIL.
*Thread Reply:* you should design it as I mentioned, using i18n, and passing the key in as an env var
*Thread Reply:* you can then also (in your env), do something like
ENV.fetch('LANG, 'en')
*Thread Reply:* so you don't need to pass in a language, it'll just default to en
*Thread Reply:* I can’t find the definition for TIL, but I think I understand what you mean by it - no default repeated input or something like that. Doesn’t that mean Background breaks TIL, though? Isn’t that for Steps and such that are supposed to be repeatable for all tests in a Feature?
*Thread Reply:* Sorry TIL or TIP is test isolation law/principle. which states tests should be independent.
*Thread Reply:* you shouldnt have 1 test that does 2 things, it should be 2 tests.
*Thread Reply:* background and examples are just examples of DRYing, they don't repeat things inside a single test.
*Thread Reply:* Just simply do something like I said earlier
Feature: Language test
Scenario: Welcome message
Given I am on the homepage
Then I am welcomed home
*Thread Reply:* Then why is using the same Examples for every test TIL? Sorry, trying to understand the differentiation. And maybe I haven’t described my use case very well.
*Thread Reply:* I'm not sure what you're complaining about. Like what is the issue.
*Thread Reply:* i've told you the two ways you should architect the thing. Until you go away and build one of those two options, try not to focus on complaints.
*Thread Reply:* Sorry, not trying to complain, just to understand and develop. I’ve tried the configuration methods, but they don’t fulfill the requirements. Maybe I’m doing that wrong, but if so I haven’t yet understood the right way to do it. If so, I apologize if I’m missing something that seems obvious. I’m not sure how else to describe my issue, though, but I’ll try again:
- I have no control over how translations are done. The platform is a third party platform. Most of the users of the platform are pro bono and trying to help people who really need the help. The whole point of my testing framework is to allow people using that third party platform to run end-to-end tests. I can’t affect how the platform handles translations and I still want to help people who are struggling to file restraining orders and such and help them be safe.
- The tests are language agnostic - users are not testing anything about the text the humans see. Unfortunately, one of the ways to handle translations in that platform [which most users use] is to make .xlsx files that translate whole blocks of YAML code. The blocks of code contain the strings end users see, but they also contain code. Sometimes translators accidentally translate the code and break things. That is, the human language isn’t the only thing that’s affected. Here is a new improved description of the method of translation. Look at the last YAML block and the last row in the spreadsheet. That shows how code can be part of the text the translators edit.
- Users want to be able to test that the translators didn’t break anything, so after they get the code translated, they want to rerun the same exact tests, but now using the different translations.
- The tests are exactly the same, so rewriting them is complete duplication, which is also important to avoid. Sometimes people are translating 5 languages. If they had 10 tests (which I’d hope they’d have more), writing a different test for each language would mean 50 tests to maintain.
- The users want to usually test just the default language because running tests in all languages every time will make the tests take
ntimes longer, wherenis the number of languages they’re translating. [In some cases, that can take hours.] Just after translation, the uses want to be able to test multiple translations in one go or a single language at a time. They want to run them as separate tests so they can isolate which translation has a mistake in it. Does that help at all? Sorry if I’ve repeated stuff you’re already clear on, I’m just not understanding how the given configuration methods can help fulfill the requirements I’m describing.
*Thread Reply:* From what I can tell, using the configuration methods described, I can allow users to test one language per test run. Because of Reasons that are equally as frustrating, if not more frustrating, than the translation method, users would have to wait for each test suite to finish before triggering the next one as opposed to just triggering them all at once. It’s a lot of babysitting for 5+ languages. If I have to, I’ll make users do that, but that’s not the ideal choice.
*Thread Reply:* Again, because of the Reasons, users will have to run each set of language tests in series if they’re running them using the configuration method. Or at least, I can’t see a way to avoid it. When the users are running the tests in GitHub, each time the action runs, the tests have to run their setup. Each time the tests run their set up, no other tests can be running at the same time.
*Thread Reply:* Maybe there’s something I’m not understanding about how to use the configuration and still avoid that.
*Thread Reply:* I’m lost now. Just speak to your QA lead or something and have them figure this out. I’ve literally given you most of the solution
*Thread Reply:* I’m the whole development team. Sorry to have frustrated you. Just so I understand, do you feel the configuration methods cover the requirements? Have I missed how the configuration methods work?
*Thread Reply:* I’ve given you both options for solving the prob using a translation file. And if you need to be verbose you already have the soln
*Thread Reply:* Maybe if I explain what I’ve understood about the configuration methods.
*Thread Reply:* User sets an env var to the language. LANG=es. They run the tests once with that language. They then set the env var to a different language. LANG=nb. They run the tests again with the new language. Is that correct?
*Thread Reply:* How they set it is irrelevant. Just that it is set using an env key.
*Thread Reply:* Write a shell loop or pass in a bar that is comma separated and unchain it
*Thread Reply:* Wow, loop in the shell instead. That could work I think. I can at least try it. I’m not sure what a bar is, but I don’t think I should unchain it (right?), so I probably want to skip that option anyway. Anyway, I can look up a bar.
*Thread Reply:* then ```# loop_cucumber.sh
languages=$(echo $LANG | tr "," "\n")
for locale in $languages do LANG=locale cucumber done```
*Thread Reply:* You might consider giving the pretty formatter a try https://github.com/cucumber/cucumber-js-pretty-formatter
*Thread Reply:* I am not sure if I understand correctly the answer but if you want to know how cucumber(js) works you should look at source code.
A good start might be in cli.run() function
https://github.com/cucumber/cucumber-js/blob/main/src/cli/index.ts#L41
Also the gherkin language's model diagram will help you understand how things work
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* We could go really deep on this.
What problem are you trying to solve?
*Thread Reply:* I was looking for connection between feature files and the step.js files...
*Thread Reply:* In this section of the intro docs, it describes how to get started. It gives an example of code in steps.js. The feature files use the sentences that you create in your steps.js.
*Thread Reply:* Hello @Terence Tian, there is a variable called CUCUMBERWORKERID that is set to 0, 1, 2 etc helping you distinguish your workers.
From thereon my advice would be to have different .env (ex: .env.0, env.1) files for each worker that has a set of different URLs and other needed configs.
You can load your dotenvs with something like:
require('dotenv').config({ path: `.env.${process.env.CUCUMBER_WORKER_ID}` })
https://github.com/cucumber/cucumber-js/blob/main/docs/parallel.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* @Aleksandar KIRILOV thanks for the advice, I end up using spawn in child_process to distribute work and env variables to different workers to run cucumber.js command
*Thread Reply:* For item 2, I think you can use tags on feature files and use tag expressions to only test those features.
*Thread Reply:* Re running a subset of scenarios, you have a few options including tags as mentioned above https://github.com/cucumber/cucumber-js/blob/main/docs/filtering.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* There’s no way to capture the publish message unfortunately. Probably the best thing would be to direct stderr output to a file when you run, and then have another build step that parses it from that file.
*Thread Reply:* For context, the future of the Reports service is uncertain and we’re not currently making any changes around it. If you can get by with the html formatter (which has roughly the same outut) then I’d probably recommend that.
*Thread Reply:* emm, I understand the returned message is out of the control of cucumber-js. Looks like the only option is patch by myself and grab the url for now.
*Thread Reply:* The stderr output is too big which is hard to parse. and html formatter is what we are using but we are migrating to GitHub actions that doesn't have a good storage solution for it. Instead of setup my own file storage in cloud, I feel the cucumber.io report is simple.
*Thread Reply:* You might be using import syntax in your TypeScript, but when it’s compiled down to JavaScript by default it is transformed to require syntax. You can change this in your tsconfig.json in the compilerOptions though.
*Thread Reply:* Feel free to pop your config in here and I can sanity check. It’s somewhat tricky to get right, and the picture is complicated by the import vs require thing.
*Thread Reply:* Ok thats probably the issue, here is the config
{
"compilerOptions": {
"target": "ES2022",
"module": "CommonJS",
"strict": true,
"allowSyntheticDefaultImports": true,
"resolveJsonModule": true,
"allowJs": true,
"esModuleInterop": true
}
}
*Thread Reply:* I meant your cucumber config, but yeah the ‘module’ option will need to change to something like ‘esnext’
*Thread Reply:* Issue fixed with Process.env[NODETLSREJECT_UNAUTHORISED”]=0
*Thread Reply:* Thank you, your reply was very helpful. [For the People of the Future, the problem was that I was trying to manipulate values in the callback argument in After() , scenario.result.status. Shouldn’t be doing that.]
*Thread Reply:* Is it possible to manipulate the willBeRetried value in the options that are given to the After() callback, or should we avoid that too? It would be incredibly useful for us if we can choose dynamically whether to rerun the test or not.
*Thread Reply:* Same problem really, that should be immutable. I think finer-grained control of retries is an interesting use case though, feel free to raise an issue on there. It might be something we could look at with the new plugins concept.
*Thread Reply:* This comment in GitHub shows a function being handed in as an argument to which you can pass, among other things, the status of the test. Is that the second argument? What are the other arguments that are possible to hand in? Or is that an old API that doesn’t exist anymore? One example given:
After((world, callback) => {
callback(null, 'skipped');
});
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* It’s tricky to understand, but the options object is optional. So your actual hook function could be the first or second arg to Before
*Thread Reply:* I need to double check in terms of arguments to the hook function.
*Thread Reply:* Fixed the doc just now, thanks for pointing out. It’s an optional prop on the options object on both Before and After
*Thread Reply:* Consider using setDefinitionFunctionWrapper - you can catch the error that results from the step, then decide if you want to rethrow or swallow it https://github.com/cucumber/cucumber-js/blob/main/docs/support_files/api_reference.md#setdefinitionfunctionwrapperwrapper
*Thread Reply:* I’m not sure that’ll do what we want. We want to throw it initially - fail the Step - but catch it in After() - pass the Scenario.
*Thread Reply:* I have a similar test, maybe it's not the best way but it works for me https://github.com/tim-yao/cucumber-nightwatch/blob/main/test/features/cucumber_nightwatch.feature
<a href="https://github.com/tim-yao/cucumber-nightwatch">tim-yao/cucumber-nightwatch</a>
*Thread Reply:* I haven’t been able to find where your code is that implements those steps, @Tim Yao, though thank you for the example and I’m very interested to see that.
*Thread Reply:* Remember that I want the step to fail, but the scenario to pass. That means the failure has to happen in After()
*Thread Reply:* Sorry, I mean After() has to swallow up the failure - make sure the Scenario as a whole passes.
*Thread Reply:* Step definition can be found in https://github.com/tim-yao/cucumber-nightwatch/blob/main/test/features/cucumber_nightwatch.steps.js
<a href="https://github.com/tim-yao/cucumber-nightwatch">tim-yao/cucumber-nightwatch</a>
*Thread Reply:* But I don’t want the failing step to pass. I want the failing step to fail. I then want the scenario to pass.
*Thread Reply:* Thank you for pointing me to the code you were talking about, though. I think it just doesn’t achieve what I want to achieve.
*Thread Reply:* The code I posted is exactly what you asked for. But I think you didn't read the code though. The test is testing the failure of a sub-process of another cucumber test. If it gets the failed test result, then it passes. However, you have to read the code then you can understand. It's quite hard to explain all the details.
*Thread Reply:* Wow, yeah, I didn’t understand the full context. I wasn’t really familiar with all of this stuff. Your clarification helped a lot, thank you.
*Thread Reply:* https://github.com/Tallyb/cucumber-playwright Have you tried this one?
*Thread Reply:* I did a comparison with a working HTML report. The broken HTML report stops at before the testRunFinished part. Without continue output the rest of the HTML.
*Thread Reply:* Another different formatter error we got yesterday, after all test run finished:
/app/node_modules/@cucumber/cucumber/src/formatter/helpers/summary_helpers.ts:77
objects.forEach((x) => (counts[x.status] += 1))
^
TypeError: Cannot read properties of undefined (reading 'status')
at /app/node_modules/@cucumber/cucumber/src/formatter/helpers/summary_helpers.ts:77:36
at Array.forEach (<anonymous>)
at getCountSummary (/app/node_modules/@cucumber/cucumber/src/formatter/helpers/summary_helpers.ts:77:11)
at formatSummary (/app/node_modules/@cucumber/cucumber/src/formatter/helpers/summary_helpers.ts:53:23)
at PrettyFormatter.logSummary (/app/node_modules/@cucumber/cucumber/src/formatter/summary_formatter.ts:72:20)
at EventEmitter.<anonymous> (/app/node_modules/@cucumber/cucumber/src/formatter/summary_formatter.ts:27:14)
at EventEmitter.emit (node:events:525:35)
at EventEmitter.emit (node:domain:489:12)
at Coordinator.onWorkerProcessClose (/app/node_modules/@cucumber/cucumber/src/runtime/parallel/coordinator.ts:200:29)
at ChildProcess.<anonymous> (/app/node_modules/@cucumber/cucumber/src/runtime/parallel/coordinator.ts:158:12)
*Thread Reply:* Both HTML reports stoped at before the testRunFinished part.
*Thread Reply:* Hello in my opinion if you need it to continue the these are two separate scenarios. Maybe you should split them into smaller ones
*Thread Reply:* You could set your interval id on the world (this) and then clear it from a AfterStep hook
*Thread Reply:* https://github.com/cucumber/compatibility-kit/blob/main/.github/workflows/release-ruby.yml#L36-L38
<a href="https://github.com/cucumber/compatibility-kit">cucumber/compatibility-kit</a>
*Thread Reply:* You can https://github.com/cucumber/cucumber-js/blob/main/docs/retry.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
Hi everyone! I'm integrating Cucumber in an existing React app testing it with react-testing-library. It's working, but I'm having issues with some external libraries and I would like to know if anyone faced this kind of issue before and what you all can recommend to solve it. I'm searching about how I could mock libraries or React components but they all use Jest, is there a better approach for this or do you suggest go with Jest? (I was looking at the example provided in this comment).
Thank you all!
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
Hi all, could you please help with this?
*Thread Reply:* Solution here: https://stackoverflow.com/questions/77356810/run-cucumber-by-tag-or-run-all-tests?noredirect=1#comment136374867_77356810
*Thread Reply:* Weird. We don’t have a config, but way back when we messed with the package.json script our tags didn’t give us problems. I’m not the one that wrote that code, though.
<a href="https://github.com/SuffolkLITLab/ALKiln">SuffolkLITLab/ALKiln</a>
*Thread Reply:* That’s odd!
Do you have any expensive setup operations that run in a BeforeAll hook or somewhere else?
*Thread Reply:* No particularly heavy pretreatment. We have beforeStep functions, but the execution time doesn’t seem to be affected. I feel that it’s more the compilation that takes time.
*Thread Reply:* Okay, it would be good to profile this to see where the time is going. I need a refresher on how best to do that in Node.js though!
*Thread Reply:* It’s definitely related to compilation, when I run with the environment variable TS_NODE_TRANSPILE_ONLY=true (so no compilation) I have a total execution time of 9s! But we lose a little the interest of TS if we generalize this…
I am very interested in having a view of what is happening in compilation, I have not found a solution yet.
*Thread Reply:* Ah interesting. Do you see that if you compile separately with tsc?
*Thread Reply:* For past projects I have set ts-node to transpile only but then added a build step in CI to compile with tsc as a compromise.
*Thread Reply:* I don’t think so unfortunately. Playwright would need a way to preprocess features into tests and it doesn’t have that as far as I can see.
For Cypress there is https://github.com/badeball/cypress-cucumber-preprocessor
*Thread Reply:* Node.js executes JS code. Either you need to compile TS code into JS, or you need to use ts-node to execute TS code in runtime.
*Thread Reply:* I probably don’t know enough to help, but you probably need to link to your code for anyone to really try to understand what’s going on.
*Thread Reply:* try npm run test -- --condition=true
https://stackoverflow.com/a/14404223
*Thread Reply:* Before hooks are always executed once per scenario.
BeforeAll hooks are executed once per test run in serial mode, or once per worker in parallel mode. Since you have parallel: 5 this will mean 5 times in practise. You could use BeforeAll without parallel, or (better) you could make your purchaseCampaign function idempotent so when called a 2nd, 3rd time etc it will just no-op since the record is already created.
*Thread Reply:* @David Goss How can I use BeforeAll without parallel? I can only set timeout for BeforeAll. Are there other ways I am missing? The credentials from the AWS Parameter store are being retrieved 5 times. And the campaign was also purchased 5 times, but I need it to be purchased once. I'd appreciate your help.
*Thread Reply:* If you want the BeforeAll only run once, you need to not use parallel
*Thread Reply:* or as mentioned, have it do some sort of halting execution.
create unless already_created? so to speak
*Thread Reply:* Is there any way to make BeforeAll hook run once? Like, maybe some passing environment variables?
*Thread Reply:* Some environment variables get set by Cucumber on the worker processes, see https://github.com/cucumber/cucumber-js/blob/main/docs/parallel.md
So, you could do something like if (process.env.CUCUMBER_WORKER_ID === '0') to make your thing only run on the first worker.
But as noted previously, making your setup functions of a “create if not exists” nature will give you a better time in the long run.
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* If you want to retry failed tests, you can take look at this part of documentation: https://github.com/cucumber/cucumber-js/blob/main/docs/retry.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* I already retry failed tests. I want more information about the failed tests when a new test runs.
*Thread Reply:* You can create an object or an array. Then push the result of the scenario to that object or an array. Or you can also write results to json file so that you read it after runtime.
*Thread Reply:* Sorry if I’m off base here, but what part of that is the unique id for the Scenario that I’d be able to connect to a later attempt?
*Thread Reply:* I need to be able to recognize in the retry that the current retry is connected to that specific failed scenario.
*Thread Reply:* You can’t do this at the moment. I am going to be re-looking at retry though with a view to making it a more generic extension point where you can plug in your own retry logic.
What kind of context would you want to pass from the failed attempt to the next attempt, out of interest?
*Thread Reply:* Good question and hard to answer because I haven’t gotten to experiment with anything. In my specific instance, I’d want the random seed I’d created in that previous Scenario. To generalize that, maybe what I want is to be able to communicate between tests. I suppose I might be able to use the global scope for that. It would help with retries, but even in parallel tests, could see this kind of info being useful as input for all running tests. For example, I have one case where if one test with a specific tag fails, none of the other tests that have that tag should be run either because they’re all going to fail and it wastes a lot of time. Maybe these things really are something for me to handle in the global scope somehow. Is that possible? And does that global scope exist for parallel tests too? I have to admit I haven’t tried those yet.
If I were trying to brainstorm more generally, for retries specifically people might want the info from the scenario that failed. That might be enough for most people.
*Thread Reply:* Why do you want both Cucumber and Jest? They’re both test runners but you can only really use one of those at a time
*Thread Reply:* > in combination with Jest for rendering the components If you want the part that renders a React component, that’s not Jest doing that. It’s maybe testing-library with a JSDOM environment, or something similar. You should be able to use that within Cucumber step definitions.
*Thread Reply:* In your docker compose command you have
--build-arg TAGS="-- --tags "not @bug""
As your usage is enclosed in double quotes, you may need to escape the inner ones that wrap your tag expression.
*Thread Reply:* @David Goss Need your help here again. When I run tests by tag with this command locally it worked fine:
cucumber-js --config=configs/cucumber.cjs --profile ci --tags "@3 and @4 and @5"
But when I ran exactly the same command from Jenkins, it did not work. Tags are passed as arguments to docker. Docker image is created in Linux ec2 instance. I've been trying to solve this issue for more than a month already, but no luck. Could you enlighten me, please?
*Thread Reply:* Update: if tags are passed with comma without space between, the test run started but no tests were executed.
*Thread Reply:* You see that error because your file is calling setDefaultTimeout, but you’re only supposed to do that when an actual test run is happening.
If you split out your cleanup functions into another file that doesn’t call any Cucumber functions, and then run that, you should be fine.
*Thread Reply:* The cleanup function is in another folder:
Project folder:
|-> hook.ts // This is where setDefaultTimeout is.
|-> src
|-> utils
|-> clearData.ts
clearData.ts has few imports but none of them are related to Cucumber (either step definitions or POMs).
clearData.ts:
```import { AxiosError } from 'axios';
import { DashboardPayload } from '../payloads';
import { httpRequest } from './api/httprequest/httpsRequest';
import { ListOfCampaigns } from './interfaces';
import { RequestBuilder } from './api/httprequest/RequestBuilder';
import { logger } from './logger/logger';
export async function deleteDraftCampaigns() { const { url } = DashboardPayload.search.admin; const config = new RequestBuilder() .setUrl(url) .setMethod('GET') .setCookies('admin');
const { data: { data: listOfCampaigns } } = await httpRequest<ListOfCampaigns>(config);
logger.info(Found ${listOfCampaigns.length} campaigns);
const filteredCampaigns = listOfCampaigns.filter(campaign => campaign.campaign_status === 'draft' && campaign.name.includes('Test Automation'));
logger.info(${filteredCampaigns.length} campaigns selected for deletion);
if (filteredCampaigns.length > 0) {
for (const campaign of filteredCampaigns) {
const deleteRequest = new RequestBuilder()
.setUrl(/api/campaign/${campaign.id}/delete)
.setMethod('POST')
.setCookies('admin');
try {
const { status } = await httpRequest(deleteRequest);
<a href="http://logger.info">logger.info</a>(`Deleting campaign ${campaign.id}... status ${status}`);
} catch (error) {
const err = error as AxiosError;
if (err.status! >= 500) {
<a href="http://logger.info">logger.info</a>(`Deleting campaign ${campaign.id}... status ${err.status}`);
}
}
}
} else { logger.info('No campaigns for deletion'); } }```
*Thread Reply:* The stack trace shows your hooks.ts file being executed
*Thread Reply:* I tried by commenting out setDefaultTimeout, the same result:
*Thread Reply:* Same applies to all cucumber functions, you see it has highlighted BeforeAll now
*Thread Reply:* The message re formatter notation is only a warning and isn’t related to the error you have below it. Seems like a hook or step issue.
*Thread Reply:* When you say log, are you doing this.log (which emits a plain text attachment to formatters) or like console.log?
*Thread Reply:* console.log. I didn’t know this.log was a thing. Also, console.log gets things to appear in the GitHub actions console [while the action is running].
*Thread Reply:* I suspect console.log() is async. Despite that, it works well for all the Scenarios after the first one.
*Thread Reply:* Let’s try and put together a minimal repro. Can you share the code from that Before hook?
*Thread Reply:* Great idea, not sure why I didn’t think of it, though I’m not sure we can use my code for that. Maybe a fresh repo. These are the relevant lines.
*Thread Reply:* Is there an example of, say, the simplest repo ever? With a github action to run the tests? If so, I could fork that and try to recreate there.
*Thread Reply:* I found a stackoverflow suggesting to use try {...} catch () {...} finally { await browser.close() }, but I’m not sure how to do that with cucumber. Would I put this in the Before() function, where I first open the browser? Do I put it in the After() somehow? Would that even help?
*Thread Reply:* But you can use today in your feature files (works the same as And)
*Thread Reply:* If and Else would be control flow, that’s not something that will ever be added to the syntax
*Thread Reply:* what about other custom keywords? Is it possible to add/register them myself?
*Thread Reply:* Just wanted to insert my custom keywords, if possible, to make scenarios look good.
*Thread Reply:* This sounds like connextra documenting that can be done in between feature and your first scenario invocation.
Here you can write any words you want. Maybe that will help ?
*Thread Reply:* ```Feature: Eating cucumbers
It is well known that cucumbers count as one of your five a day but what people don't often think of is that a cucumber salad if made correctly can be very tasty
As a healthy eating enthusiast I want to provide a quick and easy way for people to see recipes for cucumbers So that I am able to encourage more people to cook and prepare meals with cucumbers And I am able to make more money selling my cookbooks.
Scenario: .......
Rule: Cucumbers can be grown in cold climates Scenario: ......```
*Thread Reply:* Additionally I noticed that if portfolioTitle has single word, then the application works well but if portfolioTitle has multiple words then I get the above error.
*Thread Reply:* First, can you show us what the Scenario Outline itself looks like? Also, did you copy/paste anything into this Examples table any point? I've seen weird things like this happen before with zero-width spaces from webpages, you can't see them but they can cause real havoc. Maybe try deleting the whole Examples section and re-type it, then see if it still behaves the same way.
*Thread Reply:* Hi @Steven Hunt, Thank you for your help. I did copy paste it but from another example.
Right now, the only workaround I had was using a single word. This is the scenario. I tried retyping the whole scenario section but it wasn’t helpful.
*Thread Reply:* I would recommend writing your step definition like this: "enters value in the respective field with {string} {string}" and then there would be double quotes around each parameter like this in your feature file:
Then enters value in the respective field with "<artistFullName0>" "<portfolioTitle>"
It's the step definition where your parsing is getting messed up.
*Thread Reply:* Thank You @Steven Hunt . I added double quotes and “{string}” in the step definition file as well and it worked.
*Thread Reply:* Quick tip here: You used examples, not data tables. Data table values are passed as a 2D array to the last parameter of the anonymous function of the step definition.
Then I check the following data
| First name | John |
| Last name | Smith |
Step definition:
Then('I check the following data', function(dataTable) {
const rows = dataTable.raw();
// [ ['First name', 'John'], ['Last name', 'Smith'] ]
});
*Thread Reply:* If you've async function that takes a screenshot or records a video and then attaches it, most likely it is the cause. I had a similar issue, and resolved it by adding setTimeout after async function.
Has anyone successfully integrated CucumberJS with Qase. I know Qase has a reporter library that supports it https://github.com/qase-tms/qase-javascript/tree/master/qase-cucumberjs#readme but it seems to be broken
*Thread Reply:* From the allure-cucumber-js readme:
> If you want to retain default formatter add some dummy file as output:
So you want to do something like:
format: [
'progress',
[`file://{path.resolve(__dirname, 'allureReporter.cjs')}`, './dummy.txt']
]
This will direct progress output to the terminal, and then a no-op file will accept the allure output while it does its normal thing.
*Thread Reply:* (I’m not a user of allure so anything beyond this, you would need to consult that project.)
*Thread Reply:* Docs on how to specify formats in the config (which has changed a bit hence the nested array in my example there) https://github.com/cucumber/cucumber-js/blob/main/docs/formatters.md
<a href="https://github.com/cucumber/cucumber-js">cucumber/cucumber-js</a>
*Thread Reply:* No, but you might find Background useful https://cucumber.io/docs/gherkin/reference/#background
*Thread Reply:* Hard to say without more details, but usually this is a signal that your scenarios aren’t fully independent and are competing for a resource. It sounds like you have Before and After hooks to setup and teardown resources though which is the right pattern. Are you using the parallel mode?
*Thread Reply:* Thanks David. No we are not using the parallel mode.
*Thread Reply:* > one more information is in beforehooks our library which needs the usbserial binds and in afterhooks it unbinds
Could this be causing an issue if you are binding and unbinding multiple times in a single process? You might consider using BeforeAll and AfterAll instead if it’s applicable
*Thread Reply:* Error: Webpack Compilation Error
Module parse failed: Unexpected token (3:5)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See <https://webpack.js.org/concepts#loaders>
| Feature: Analytics
| Scenario: SingleDownload
> When the user is in A&D page
| Then perform single download
at Watching.handle [as handler] (/Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/@cypress/webpack-preprocessor/dist/index.js:212:23)
at /Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/webpack/lib/Watching.js:303:9
at Hook.eval [as callAsync] (eval at create (/Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (/Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/tapable/lib/Hook.js:18:14)
at Watching._done (/Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/webpack/lib/Watching.js:299:28)
at /Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/webpack/lib/Watching.js:213:21
at Compiler.emitRecords (/Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/webpack/lib/Compiler.js:919:5)
at /Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/webpack/lib/Watching.js:191:22
at /Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/webpack/lib/Compiler.js:885:14
at Hook.eval [as callAsync] (eval at create (/Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (/Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/tapable/lib/Hook.js:18:14)
at /Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/webpack/lib/Compiler.js:882:27
at /Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/neo-async/async.js:2818:7
at done (/Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/neo-async/async.js:3522:9)
at alreadyWritten (/Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/webpack/lib/Compiler.js:714:8)
at /Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/webpack/lib/Compiler.js:802:19
at /Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/node_modules/graceful-fs/graceful-fs.js:123:16
at /Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/graceful-fs/graceful-fs.js:123:16
at /Users/atul/Library/Caches/Cypress/13.8.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/graceful-fs/graceful-fs.js:123:16
at FSReqCallback.readFileAfterClose [as oncomplete] (node:internal/fs/read/context:68:3)
*Thread Reply:* This means that you haven't configured webpack properly. I recommend looking at any of the webpack examples at https://github.com/badeball/cypress-cucumber-preprocessor/tree/master/examples
Hi everyone! I run 5 tests in parallel. At the beginning, test run in 5 different browsers, and then at some point test won't run parallelly, but only one test at a time is executed. Do you know why cucumber switches to executing tests one by one?
*Thread Reply:* Most definitely not. There are retries tags on few scenarios only.
